Non-Functional Testing

Question 1

Does testing only touch upon functional behaviour?

Answer 1

No! You can test on many different non-functional qualities:

• Usability
• Performance
• Reliability
• Maintainability
• Portability
• Security

Question 2

What are some testing targets if we are focusing on performance?

Answer 2

Does the system work well in its environment?
- Respond quick enough
- respond under reasonable load?
- use reasonable system resources
...

Question 3

What is a reasonable load?

Answer 3

Difficult to determine! This is always changing and you will have to adapt.

Question 4

What is Ratcheting?

Answer 4

It allows progressive performance enhancements!

Take a test suite, take a subset of that suite and measure off of a threshold.

Lower that threshold until you break the system!

Question 5

What are ways to simply test performance?

Answer 5

simulate realistic loads:

• Goldilocks Principle: not too fast, not too slow.
• realistic is determined based on passed experiences and estimates.

Stress testing:
- quantifying conditions under which a system (or its components) will break.

Steal ideas from other disciplines!
- look at research relating to human attention span for example, and conform to that.

Question 6

Define stress testing!

Answer 6

High load
Sustained load
Sudden spike of load
All of the above.