08 Software Non Functional Requirements Flashcards
what are the coverage metrics
- statement coverage
- detect bug in a statement only by executing statement - branch coverage
- all branch in software should be executed once - path coverage
- all execution path should be executed once
what is statement coverage
- assess quality of test by measuring how each of code statement is executed
- target 100%
- statement coverage = number of executed statements/ total number of statements
what is branch coverage
- test all possible branches
- branch coverage = number of executed branches/ total number of branches
- 100% branch coverage does not mean that all bugs are detected
what is path coverage
- test all possible paths
- path coverage = number of executed paths/ total number of paths
- total independent path calculated by cyclomatic complexity number
- path coverage leads to more thorough testing
Drivers and stubs
driver calls (module under test) calls (stub)
drivers simulate module that calls the module under test
stubs simulates the behaviour of component used by the module under test
integration test strategy
- integration plan specify order to combine modules into partial system
- different approaches
- top down
- bottom up
- sandwich
- big bang
top down approach
- start with root module
- test using stubs to replace missing modules
- 1 driver to call root module
- add one or more called modules one by one and provide new stubs
- continue until all modules are tested
top down approach pros/cons
pros
1. early verification of high level behaviour
2. only one driver is needed
3 modules can be added one at a time
4. supports both depth first, breadth first approach
cons
1. need many stubs
bottom up approach
- reverse of top down
- start with end module, add driver
bottom up pros/cons
pros
1. early verification of low level behaviour
2. no stubs needed
cons
1. root tested the last
sandwich approach
- comprise of bottom up and top down
- simultaneously begin bottom up and top down and meet at predetermined part in the middle
sandwich pros/cons
pros
1. parallel testing
2. time saving
3. suitable for large systems
big bang approach
- every module tested in isolation
- after all modules tested, then integrate together at once and tested
big bang pros/cons
pros
1. no driver or stub is needed
2. minor planning
3. suitable for small systems
cons
1. hard to isolate bugs
functional requirements
- how system should react to input
- what inputs should it accept
- what output should it provide
- what data should it store
- what computation will it perform
non functional requirements
- quality of software
- essential to ensure usability and effectiveness. If not met, system is useless
- imprecise non functional requirements are hard to verify
- no clear distinction between functional/non functional
non functional requirement testing
- describes emergent properties of overall system that must be measured and satisfied
- cannot typically be checked for in unit test or statistical analysis
- can only be tested at system with implementation of prototype or system
information needed by non functional testing
- property of system to be measured
- measurement metrics to use
- operating conditions
- threshold system behaviour must exceed to satisfy requirement
quantifying NFR
- needs to be measurable
- values are not randomly specified(rational)
- precise values are unlikely to be know at the start
measures of reliability testing
- probability of number of failure in specified time
- probability of failure on demand
- each time system is accessed - mean time to failure
- avg time between failure - down time
- in a year
security testing
- for system operating in hostile environment
- ability to resist unauthorised attempt at usage
- continue to provide service under DOS
security testing metrics
- time and resources
- unpatched vulnerability per line of code
- patches issued per year
- successful penetrations per year
- attack surface exposure
- lifespan of password
- encryption level
security testing methods
Vulnerability testing
1. sql injection
2. port scanning
3. fuzz testing
- supply malformed input
Penetration testing
Performance testing
1. through put
2. demand
3. response
software usability metrics
- learnability
- proportion of functionalities to be mastered after training - operability
- capability to enable user to control it - likability
- user satisfaction
- satisfaction ratio
measure of testability
ability to detect, isolate, fix defects
- time to run test
- time to setup test environment
- probability of visible failures
- test coverage
