08 Software Non Functional Requirements

Question 1

what are the coverage metrics

Answer 1

1. statement coverage
   - detect bug in a statement only by executing statement
2. branch coverage
   - all branch in software should be executed once
3. path coverage
   - all execution path should be executed once

Question 2

what is statement coverage

Answer 2

1. assess quality of test by measuring how each of code statement is executed
2. target 100%
3. statement coverage = number of executed statements/ total number of statements

Question 3

what is branch coverage

Answer 3

1. test all possible branches
2. branch coverage = number of executed branches/ total number of branches
3. 100% branch coverage does not mean that all bugs are detected

Question 4

what is path coverage

Answer 4

1. test all possible paths
2. path coverage = number of executed paths/ total number of paths
3. total independent path calculated by cyclomatic complexity number
4. path coverage leads to more thorough testing

Question 5

Drivers and stubs

Answer 5

driver calls (module under test) calls (stub)

drivers simulate module that calls the module under test

stubs simulates the behaviour of component used by the module under test

Question 6

integration test strategy

Answer 6

1. integration plan specify order to combine modules into partial system
2. different approaches
   - top down
   - bottom up
   - sandwich
   - big bang

Question 7

top down approach

Answer 7

1. start with root module
2. test using stubs to replace missing modules
3. 1 driver to call root module
4. add one or more called modules one by one and provide new stubs
5. continue until all modules are tested

Question 8

top down approach pros/cons

Answer 8

pros
1. early verification of high level behaviour
2. only one driver is needed
3 modules can be added one at a time
4. supports both depth first, breadth first approach

cons
1. need many stubs

Question 9

bottom up approach

Answer 9

1. reverse of top down
2. start with end module, add driver

Question 10

bottom up pros/cons

Answer 10

pros
1. early verification of low level behaviour
2. no stubs needed

cons
1. root tested the last

Question 11

sandwich approach

Answer 11

1. comprise of bottom up and top down
2. simultaneously begin bottom up and top down and meet at predetermined part in the middle

Question 12

sandwich pros/cons

Answer 12

pros
1. parallel testing
2. time saving
3. suitable for large systems

Question 13

big bang approach

Answer 13

1. every module tested in isolation
2. after all modules tested, then integrate together at once and tested

Question 14

big bang pros/cons

Answer 14

pros
1. no driver or stub is needed
2. minor planning
3. suitable for small systems

cons
1. hard to isolate bugs

Question 15

functional requirements

Answer 15

1. how system should react to input
2. what inputs should it accept
3. what output should it provide
4. what data should it store
5. what computation will it perform

Question 16

non functional requirements

Answer 16

1. quality of software
2. essential to ensure usability and effectiveness. If not met, system is useless
3. imprecise non functional requirements are hard to verify
4. no clear distinction between functional/non functional

Question 17

non functional requirement testing

Answer 17

1. describes emergent properties of overall system that must be measured and satisfied
2. cannot typically be checked for in unit test or statistical analysis
3. can only be tested at system with implementation of prototype or system

Question 18

information needed by non functional testing

Answer 18

1. property of system to be measured
2. measurement metrics to use
3. operating conditions
4. threshold system behaviour must exceed to satisfy requirement

Question 19

quantifying NFR

Answer 19

1. needs to be measurable
2. values are not randomly specified(rational)
3. precise values are unlikely to be know at the start

Question 20

measures of reliability testing

Answer 20

1. probability of number of failure in specified time
2. probability of failure on demand
   - each time system is accessed
3. mean time to failure
   - avg time between failure
4. down time
   - in a year

Question 21

security testing

Answer 21

1. for system operating in hostile environment
2. ability to resist unauthorised attempt at usage
3. continue to provide service under DOS

Question 22

security testing metrics

Answer 22

1. time and resources
2. unpatched vulnerability per line of code
3. patches issued per year
4. successful penetrations per year
5. attack surface exposure
6. lifespan of password
7. encryption level

Question 23

security testing methods

Answer 23

Vulnerability testing
1. sql injection
2. port scanning
3. fuzz testing
- supply malformed input

Penetration testing

Performance testing
1. through put
2. demand
3. response

Question 24

software usability metrics

Answer 24

1. learnability
   - proportion of functionalities to be mastered after training
2. operability
   - capability to enable user to control it
3. likability
4. user satisfaction
   - satisfaction ratio

Question 25

measure of testability

Answer 25

ability to detect, isolate, fix defects
- time to run test
- time to setup test environment
- probability of visible failures
- test coverage