Lecture 7 - Software Reliability

Question 1

What is software engineering?

Answer 1

● One way of looking at software is that programs are the
encoding of knowledge.
● But where does the knowledge come from?
● Information provided by clients and end-users.
● Knowledge from domain experts.
● Opinions and expertise of the programmer.
● Data from sensors.
● Outputs from learning system e.g. AI.
● SE is engineering because it is the practical application of maths and physics principles in code that HAS to work in the real world

Question 2

Why is SE important for products, processes and projects?

Answer 2

● Good software provides:
○ Competitive advantage
○ Scalability and efficiency
○ Smart systems
○ Good user interfaces
○ Flexibility

Question 3

How is software different?

Answer 3

● Software does not degrade, age or wear out.
● Every copy is identical with the same reliability.
● Unlike other engineering products software is ephemeral. It is flexible and of arbitrary complexity.
● When we find a defect in a mechanical system (for example), we repair it to its original or an in-between state.
● However, when we find a defect in a software system, we repair it to a new state.
● We analyze the reliability of a mechanical system by looking at the probability of failure of individual parts.
● However, we analyze the reliability of a software system by looking at its testing history

Question 4

Why is software different?

Answer 4

● Software has deterministic behaviour, whereas hardware behaviour is both deterministic and stochastic.
● Once the inputs have been selected the software will either execute correctly or fail.
● The nature and position of the fault may be influenced by the input, computer and/or operating system. One may translate this uncertainty into probabilities.
● A software fault is a triggering event that causes a software defect

Question 5

Why is understanding software reliability important?

Answer 5

● Very few functional objects and systems today function without software.
● As an engineer of whatever discipline you will have to
work with software and software engineers.
● It is therefore vital that you understand their world, methods and processes. You do not have to be able to drive these processes yourself but you need to know enough about them to participate effectively.
● You need to know what questions to ask

Question 6

What are some sources of risk?

Answer 6

● Software development and maintenance is a humanintensive activity
● Code bases can be large and complex (millions of lines of code)
● Some parts of code bases can be dynamic and other parts don’t change much.
● Some parts get more maintenance than others.
● Code development depends on ASSUMPTIONS

Question 7

What is requirements engineering?

Answer 7

● A significant proportion of defects arise during the Requirements analysis phase.
● A requirement is a statement about a proposed system that all stakeholders agree must be made true in order for the customers problems to be truly solved. It describes as a service or constraint, the user-level properties, general systems, specific constraints and
needs of clients.
● Requirements analysis is an incremental and iterative
process.

Question 8

What is requirements engineering standard?

Answer 8

● ISO/IEC/IEEE 29148-2018 - International Standard - Systems and software engineering – Life cycle processes – Requirements engineering.
● It defines the construct of a good requirement, provides
attributes and characteristics of requirements, and discusses the iterative and recursive application of requirements processes throughout the software life cycle.

Question 9

What are good practices for requirements statements?

Answer 9

Valid
Unambiguous
Complete
Understandable
Consistent
Verifiable
Modifiable
Traceable

Question 10

What are some assumptions?

Answer 10

● End users assume software engineers understand their domain
● Software engineers assume they understand the domain.
● Neither of these statements may be true
● Revisions to systems tend to carry forward defects from the previous system
● End users will specify what they want improved. They rarely talk about what they like about the system. Requirements always have trade-offs therefore software engineers tend to
fix the ‘bad’ things at the expense of the ‘good’ things.
● They assume what is being requested are important and those that not requested are unimportant.

Question 11

What are some software development models?

Answer 11

● There are a spectrum of models for software development for example:
● Waterfall and other software development life cycle models (SDLC)
● V model (verification and validation) - variant of SDLC
● Spiral (prototyping of Requirements phase) models
● Agile models are now the most widely used approach for conventional systems (not safety critical)

Question 12

GOING THROUGH EACH MODEL

Answer 12

Refer to slides for visual and content as they are required together

Question 13

What is softwre reliability?

Answer 13

● Software reliability may be defined as the probability of
failure-free software operation for a specified period of time in a specified environment.

● Software failure may be due to issues such as, but not limited to, errors, ambiguities, over-sights, and misinterpretation of specifications.

Question 14

What is verification and validation?

Answer 14

Verification- Am I building the system correctly?
Validation - Have I built the right system?

Question 15

What is pre-testing?

Answer 15

● We need to do verification on requirements statements, design statements, test statements BEFORE they are coded. This is done by Review Teams.
● There are many levels of testing (see the V model discussed earlier) and many different techniques that can be used once coding has commenced.

Question 16

What are the methods for verification of code?

Answer 16

● There are many methods of assessing the correctness of
code. Most are based on various forms of inspections
(reviews) and testing. A few important ones are:
● Formal methods of requirements or program proving. Formal methods are widely used for safety critical systems. In formal methods piece of code can be considered as a mathematical object and is thus susceptible to proof of correctness.
● Another approach is to use multiple independent teams to build similar systems leading to parallel testing against each other or multiple independent teams to review or test the system.
● And another is to use external tiger teams to try and penetrate security or destroy a system.

Question 17

What is s reliability growth model?

Answer 17

● In reliability growth models, software is tested for a period of time, during which failures may occur.
● These failures lead to modification and the new version goes back into test.
● This cycle continues until design objectives are met.

Question 18

What are software reliability metrics?

Answer 18

● SRMs can be grouped into three categories. 4
● Time-between-failure models.
● Fault seeding models.
● Independent testing teams.

● These are necessary to quantify reliability through the number of defects. One of the hardest questions of software engineering is “when do I stop testing?”

Question 19

What is the SRM: Fault seeding model I

Answer 19

● Purpose: To assess the number of defects in the software via seeding extraneous faults.
● An independent monitor inserts defects into the code.
● Key assumptions are as follows.6
● Defects are assumed to be independent.
● Seeded defects are distributed in the software using profiles based on previous experiences.
● Indigeneous and seeded defects have equal probabilities of being detected.

Question 20

What is the SRM: Fault seeding model III

Answer 20

● NS number of seeded faults.
● nS number of intentional seed faults found.
● nF number of faults found that were not intentionally seeded.
● Provided some seeded faults are found an estimate NF
for the MLE of unseeded faults can be determined.
● Est. NF = (nF . NS)/ nS

REFER TO SLIDES

Question 21

What is configuration management when linking to automation?

Answer 21

Automation inevitably requires software, hardware, data or
settings and also physical inputs to operate.
Knowing what versions of devices, software and values in the code is vital to ensuring robust and safe systems.
Configuration management is the process and systems to
do this.

Question 22

What are the challanges of configuration management?

Answer 22

How does an operator know that the equipment is safe
to use?
How can the operator verify that the configuration of the equipment is correct?
How does the maintainer know the software version was correctly loaded?
Is there a possibility that the software could be loaded successfully but may not work under operational conditions?

Question 23

Define configuration management?

Answer 23

Is a management process for establishing and maintaining consistency of a product’s physical and functional attributes with its design and operational information throughout its life.
It is a vital part of managing autonomous systems

REFER TO SLIDES FOR VISUALS

Question 24

Why does config management need good processes and culture?

Answer 24

● Implement a framework consistent with ISO 10007
● Write a configuration management plan
● Decide which equipment needs it (it’s expensive)
● Configuration Management affects many parts of the organisation
● Awareness is key to making the process work.
● Creating awareness through training and communication
● Check regularly with audits.
● Leadership must model the right behaviour