ISD Lecture 12 Software quality assurance. Developing systems that works

Question 1

Quality

Answer 1

Quality can be hard to explain!

There is many views of what quality is.

The transcendental view

Quality is something that you immediately recognize, but cannot explicitly define

The user view

Quality in terms of an end-user’s specific goals. If a products meets those goals, it exhibits quality.

Manufacturer’s view

Quality in terms of the original specification of the product. If the product conforms to the specification, it exhibits quality

Product view

Quality can be tied to inherent characteristics of a product

Value-based view

Quality based on how much a customer is willing to pay for a product.

Software quality
An effective software process applied in a manner that creates a useful product that provides measurable value for those who produce it and those who use it.

Question 2

Cost of Quality

Answer 2

there are three types of cost: prevention, internal failure and external failure. The later an error is identified, the more costly it is.

Question 3

Software quality assurance:

Answer 3

Which basically is an independent control of software engineering activities to verify compliance with the defined software process.

Question 4

Testing

Answer 4

is the process of exercising a program with the specific intent of finding errors prior to delivery to the end user.

Question 5

Verification

Answer 5

Evaluating whether “we are building the product right?”

refers to the set of tasks that ensure that software correctly implements a specific function.

Question 6

Validation

Answer 6

Evaluating whether “we building the right product?“

refers to a different set of tasks that ensure that the software that has been built is traceable to customer requirements.

Question 7

SQA (Model)

Answer 7

What is Quality ->
Whats the cost of quality? ->

How can we prevent errors: Effective software development process,

How can we create errors: In-effective software development process,

How can we find errors: Review and test,

How can we manage SQA: Project-level & Corporate-level

Question 8

How to make a effective software process

Answer 8

making quality less dependent on individuals

Establish an supporting infrastructure,

management aspect of process,

software engineering practices,

umbrella activities

Question 9

What is a useful product?

Answer 9

Delivers the needed content:

With no (important) errors:

Satisfies explicit requirements:

Satisfies implicit requirements:

Question 10

Where does SQA adds value?

Answer 10

The software organization:

gains added value because high quality software requires less maintenance effort, fewer bug fixes, and reduced customer support.

The user community:

gains added value because the application provides a useful capability in a way that expedites some business process.

The end result is:
greater software product revenue,
better profitability when an application supports a business process, and
improved availability of information that is crucial for the business.

Question 11

The software quality dimensions

Answer 11

Performance quality,

Does the software deliver all content, functions, and features that are specified as part of the requirements model in a way that provides value to the end-user?

feature quality,

Does the software deliver all content, functions, and features that are specified as part of the requirements model in a way that provides value to the end-user?

reliability,

Does the software deliver all features and capability without failure?

conformance,

Does the software conform to local and external software standards that are relevant to the application?

durability,

Can the software be maintained (changed) or corrected (debugged) without the inadvertent generation of unintended side effects?

serviceability,

Can the software be maintained (changed) or corrected (debugged) in an acceptably short time period.

aesthetics,

perception

Question 12

Describe McCall’s Quality Factors

Answer 12

Product revision (Maintainabliity, Flexibility, Testability) ->

Product Transition (Portabability, Reuseability, Interoperability) ->

Product Operation (Correctness, Reliability, Usability, Security, Efficiency)

Question 13

How to measure quality

Answer 13

Make it specific

Use some kind of hard metrics

Question 14

What kind of dilemma is there with software quality

Answer 14

The cost

If you produce a software system that has terrible quality, you lose because no one will want to buy it.

If on the other hand you spend infinite time, extremely large effort, and huge sums of money to build the absolutely perfect piece of software, then it’s going to take so long to complete and it will be so expensive to produce that you’ll be out of business anyway.

Either you missed the market window, or you simply exhausted all your resources.

Question 15

What is good enough software and what can be a argument against good enough?

Answer 15

Good enough software delivers high quality functions and features that end-users desire, but at the same time it delivers other more obscure or specialized functions and features that contain known bugs.

. If you deliver a “good enough” (buggy) product, you risk permanent damage to your company’s reputation.

it can be negligent and open your company to expensive litigation.

Question 16

What does prevention costs include?

Answer 16

quality planning
formal technical reviews
test equipment
Training

Question 17

What does Internal failure costs include?

Answer 17

rework
repair
failure mode analysis

Question 18

What does External failure costs include?

Answer 18

complaint resolution
product return and replacement
help line support
warranty work

Question 19

What kind of negative impact can there be with management decisions on quality?

Answer 19

Estimation decisions

E.g. too little time to do a complex program

Scheduling decisions

E.g. that requirements are unclear when design starts

Risk-oriented decisions

E.g. that projects “gets surprised” and has to re-schedule continuously.

Question 20

What kind of positive impact can there be with management decisions on quality?

Answer 20

Software quality is the result of good project management and solid engineering practice.

Project management –

project plan should include techniques and activities for quality and change management

Quality control -

series of inspections, reviews, and tests used to ensure conformance of a work product to its specifications

Quality assurance -

consists of the auditing and reporting procedures used to provide management with data needed to make proactive decisions

Projects should integrate software quality assurance practices (SQA).

Question 21

Elements of SQA

Answer 21

Standards 
Reviews and Audits 
Testing
Error/defect collection and analysis 
Change management 

Education  
Vendor management 
Security management 
Safety 
Risk management

Question 22

The roles of the SQA group

Answer 22

Support -

Prepares an SQA plan for a project: How should SQA be performed in this project?

Participates in the development of the project’s software process description: How should the project work?

Control -

Reviews software engineering activities to verify compliance with the defined software process.

Audits designated software work products to verify compliance with those defined as part of the software process.

Ensures that deviations in software work and work products are documented and handled according to a documented procedure.

Question 23

The SQA Goals

Answer 23

Requirements quality.

The correctness, completeness, and consistency of the requirements model

Design quality.

Every element of the design model should be assessed by the software team

Code quality.

Source code and related work products (e.g., other descriptive information) must conform to local coding standards

Quality control effectiveness.

The SQA group and the projects will typically try to monitor SQA processes and quality using quantitative data

Question 24

What is statistical SQA

Answer 24

Continuous improvement

Collect information about errors,

identify the underlying causes,

identify the vital few causes,

remove the causes

Question 25

What is used to analyze the causes?

Answer 25

The Fishbone Diagram

Question 26

Strategic approaches for software testing

Answer 26

Before Testing - Conduct effective technical reviews

Testing order - Testing begin at the component level and works “outward” toward the integration of the entire system

Choosing the right testing techniques - Different testing techniques are appropriate for different software engineering approaches and at different points in time.

Organizating testing - Testing is conducted by the developer of the software and (for large projects) an independent test group.

Question 27

what is the order of testing

Answer 27

o Unit test -> integration test -> validation test -> system test
o Begin testing-in-the small and move toward testing-in-the-large

Question 28

Two different kinds of test

Answer 28

Verification

Validation

Question 29

Who test the software?

Answer 29

Developer

Understands the system but wil test “gently” and, is driven by “delivery”.

Independent tester

Must learn about the system, but, will attempt to break it and, is driven by quality.

Question 30

Agile methods and SQA

Answer 30

Agile methods have a less rigid and bureaucratic approach to systems development – and traditional SQA techniques are built upon ideas that makes systems development more bureaucratic and rigid, so….

Question 31

What kind of QA techniques are there

Answer 31

Static
Do not involve the execution of code (e.g. reviews. Static techniques involve examination of documentation by individuals or groups.

Dynamic
Do involve the execution of code (tests). Testing and simulation are dynamic techniques.

The waterfall model uses both static and dynamic techniques. However, agile methods mostly use dynamic techniques.

Question 32

Agile Methods with QA

Answer 32

Integrated QA

Fast QA feedback:

System metaphor

On-site customers:

Having an On-site customer is a general practice in most agile methods. Customers help developers refine and correct requirements – validation.

Consequently customer involvement in agile methods is much heavier than in waterfall development.

pair programming
Pair programming means two programmers continuously working on the same code

: refactoring
is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.

Continuous integration
a popular practice among agile methods means the team does not integrate the code once or twice.

Acceptance testing:
is carried out after all unit test cases have passed. This activity is a dynamic QA technique.

Early customer feedback:
is one of the most valuable characteristics of agile methods.

Question 33

A SQA comparison

Answer 33

Timing

Frequency

Static/dynamic

Question 34

How to use SQA

Answer 34

Many projects suffer from badly executed SQA and test.

Make sure: 
o	Quality goals defined for the project
o	Dimensions operationalized so that it can be determined whether they are achieved or not 
o	Make a proper strategy for prevention and identification of errors 
o	Validation 
o	Verification
o	Independency
o	Resources and responsibilities 
o	Statistical SQA

Question 35

Reflections and relation

Answer 35

• Only focus on software not organizational change
• Agile development might have a weakness regarding SQA  little documentation
• When comparing with the IS success model (lecture 11) – considerations about service quality is missing
• Lecture 5 ephasize the lack of concern for non-functional requirements in agile – making QA for these requirements difficult