W2: Statistical Methods

Question 1

Explain ‘SUS’.

Answer 1

System Usability Scale.
Measures subjective usability using a 5-point Likert scale.

Question 2

What are the 10 SUS questions?

Answer 2

1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this
    system.

Question 3

How is SUS evaluated?

Answer 3

Likert scale translates into numerical values, and these values are averaged.

Question 4

Why is SUS controversial?

Answer 4

• Going from agree to strongly agree is different from neutral to agree - overall score does not change
• Some people may tend to answer moderately (agree, disagree), and some may tend to answer strongly.

Question 5

What are the 3 measures of ‘central tendency’?

Answer 5

• median
• mode
• arithmetic Mean

Question 6

Which variables is ‘median’ suitable for?

Answer 6

Ordinal variables, with an ordering

Question 7

In what cases is ‘median’ more suited than ‘arithmetic mean’?

Answer 7

Skewed distributions, data with outliers

Question 8

What needs to be done for representative scores?

Answer 8

Random sampling, as different users will give different scores.

Question 9

Why is ‘confidence interval’ used?

Answer 9

To convey the quality of the obtained results.

Question 10

What does ‘confidence interval’ indicate?

Answer 10

The range in which a parameter (e.g., median) is expected to lie.

Question 11

What does ‘confidence level’ (e.g. 95%) represent?

Answer 11

Probability that the parameter is within the interval

Question 12

“Confidence intervals are always between original data points.”
TRUE or FALSE?

Answer 12

TRUE

Question 13

What is ‘significance testing’ used for?

Answer 13

Helps determine whether differences between measures are meaningful.

Question 14

Explain ‘TWO-sided sign test’.

Answer 14

• Tests if one measure is better than another for most of the population.
• Computes confidence interval for the median of the difference.
• If confidence interval excludes zero, result is significant.

Question 15

Explain ‘ONE-sided sign test’.

Answer 15

• Useful when a specific measure is expected to be better.
• Requires fewer measurements for significance.
• Interval is open on one side, making significance easier to achieve.
• Result is significant if p-value < 0.05

Question 16

Explain ‘Wilcoxon Signed-Rank Test’.

Answer 16

• Offers higher statistical power than the sign test.
• Takes differences into account and is suitable for cases with less clear results.
• Provides a w-value based on ordered differences and signs.
• The critical w-value determines significance, with lower w-values indicating significance.

Question 17

What is a ‘Type I error’?

Answer 17

False Positive
Null hypothesis is rejected when it should not be rejected.

Question 18

What is a ‘Type II error’?

Answer 18

False Negative
Null hypothesis is not rejected when it should be rejected.

Question 19

How is ‘w-value’ for ‘Wilcoxon Signed-Rank Test’ calculated?

Answer 19

1. Order differences according to their absolute size:
   -0.3, 0.6, -2.1, 2.2, 3.4, 5.1, 5.7, 6.1, 6.8
2. Get an ordered list of signs
   - + - + + + + + +
3. 1 3
   w-value = 1 + 3 = 4
4. Look up in table for critical w-value
   For n = 9, maximally allowed w-value is 5 for p = 0.05, so result is significant.