Q11

Question 1

What type of data is collected in Field studies?

Answer 1

Qualitative

Question 2

Name 2 evaluation techniques associated with field studies.

Answer 2

1. Direct observation

2. Interviews

Question 3

You are observing people using a system for the purposes of evaluation. Give one advantage and one disadvantage of:

The technique of asking people to think aloud while using the system:

Answer 3

Advantage: You can get an idea of the user’s first instincts about using the product.

Disadvantage: In order not to appear silly or foolish, users may think about what they are going to say.

Question 4

You are observing people using a system for the purposes of evaluation. Give one advantage and one disadvantage of:

Collecting data solely by an evaluator taking notes during the evaluation.

Answer 4

Advantage: Observer can document activities as they happen.

Disadvantage: Data can be missed, especially if the note-taker fails to make a good record.

Question 5

You wish to evaluate a mobile phone designed specifically for elderly people. However, you don’t have much money for evaluation so you plan to ask your four children to do the following:–

• look up a particular number in the phone’s directory and phone it. You plan to measure the time it takes each of them to do this task and find the mean.
• use it for an hour and comment on the features they particularly like and don’t like. You plan to list both positive and negative features.

Give five different ways in which your choice of participants might bias your evaluation.

Answer 5

1. The children are not beneficiaries of the system.
2. The mobile phone is being designed for elderly people – not children
3. The features the children like may not match what the elderly people like
4. Conversely features the children don’t like may appeal to the projected user group
5. It is likely that the children will not take as long to carry out the task as the target group.

Question 6

The think-aloud technique is often used in evaluation studies. Describe the think-aloud technique.

Answer 6

The think-aloud technique requires a participant to say out loud everything they are thinking and trying to do during an activity, so that their thought processes can be heard out loud.

Question 7

Consider the use of the think-aloud technique in each of the following evaluation approaches: usability testing, field studies and analytical evaluation. What kind of insights would this technique produce in each case?

• Usability testing
• Field studies
• Analytical evaluation

Answer 7

Usability testing:

Insight into users’ thought processes while they perform specific tasks in a controlled environment

Field studies:

Insight into users’ thought processes while they perform their normal tasks in their natural setting

Analytical evaluation:

In this approach, experts are the participants and so you would gain insights into the application of the analytical method rather than the product being evaluated.

Question 8

Describe the impact of having a very limited amount of money on the following evaluation methods:

• Direct observations
• Interviews
• Questionnairs

Answer 8

Direct observations; Very direct method for collecting data or information – and need not be costly, all you need is to be able to take good notes, although it does take considerable time

Interviews; can be extremely time consuming

Questionnaires. If you were really short of cash then the amount of potential participants you can reach is quite high, but the response rate may be low and the responses may not be what you want.