L9, Data collection 2 - qualitative

Question 1

Pros and cons with using available information?

Answer 1

PRO
1. inexpensive
—> data already there
2. available to see trends over the past

CON

1. difficult to access
2. ethical issues concerning confidentiality
3. information may be imprecise or incomplete

Question 2

Pros and cons with observations

Answer 2

PRO

1. more detailed and context-related information
2. tests reliability of responses in questionnaire

CON

1. ethical issues with confidentiality and privacy
2. observation bias
3. presence of data collector high influence results
4. thorough training of research assistants is required

Question 3

Pros and cons with interviewing?

Answer 3

PRO

1. suitable for use with both literates and illiterate
2. permits clarification of questions
3. has higher response rate than written questionnaires

CON
1. The presence of the interview might influence the response

2. reports of events might may be less complete than with observations

Question 4

Pros and cons with small scale flexible interview?

Answer 4

PRO:
1. permits collection of in-depth information and exploration of spontaneous remarks by respondents

CON:

1. Interviewer might influence interviewer
2. analysis of open-ended data is more difficult and time-consuming

Question 5

Pros and cons with larger scale fixed interview?

Answer 5

PRO
1. easy to analyze

CON:
1. Important information may be missed due to spontaneous remarks by respondents are not recorded or documented

Question 6

Pros and cons with Adminestering questionnaires?

Answer 6

PRO
1. Less expensive
2. permits anonymity
—> honest results
3. no need for research asistance
4. eliminates bias due to  phasing questions differently with  respondents

CON:

1. cannot be used with illiterate respondents
2. often low rate of response
3. questions may be misunderstood

Question 7

Pros and cons with participatory and projective methods?

Answer 7

PRO:

1. provide rich data
2. positive spon offs for knowledge and skills by researchers and informants

CONS:
1. require extra training of researchers

Question 8

How should you make the choice of method for a market study?

Answer 8

It should be determined by the purpose of the study
— Is it explorative?
— Confirmative?

The purpose of market studies can be assumed to differ in different phases of the product development process

Question 9

In what ways could requirements for solutions be expressed?

Answer 9

1. as descriptions of problems
2. as comparisons
3. as assumptions
4. as behaviors
5. as actions
6. as requirements
7. as solutions

Question 10

How does requirements emerge?

Answer 10

1. they do NOT emerge in a vacuum
2. In a situation
   —> “the use situation”
   —> “the use system”
3. the methods chosen must be able to grasp this system

Question 11

The choice of method is a matter of four different choices!

Answer 11

1. data collection method
2. participants
3. context
4. Mediating tool /stimuli

Question 12

Choice of participants?

Answer 12

— The qualitative dimension?
— The quantitative dimension?

Depends in the sampling:
1. statistical sampling?
2. theoretical sampling?
—> choose participants with a particular property
3. Critical sampling?
4. Convenience sampling?

Question 13

How can you divide users into groups?

Answer 13

1. The common
   —> age
   —> gender

2. Some other
—> product user experience
—> problem experience
—> interests
—> physical properties
—> personality

Question 14

Some thoughts on participants?

Answer 14

1. Think about what characteristics are really important when you decide on your sampling strategy
2. participants should be part of the identified user group
3. avoid representatives of the users
4. should preferably have experience of the use
   —>or experience of the problem

Question 15

What kind of participants could be difficult to find?

Answer 15

1. elderly frail

2. busy people

Question 16

what kind of participants are easy to find?

Answer 16

1. unemployed
2. students
3. women with academic background

Question 17

the quantitative dimension of qualitative studies?

Answer 17

1. how many
   —> the more the better
2. Find the saturation point
3. The rule of diminishing returns

Question 18

How many people should be subject for qualitative studies according to literature?

Answer 18

1. Griffin and Hauser
   —> 30 interviews = 99% requirements
   —> 98% of customer requirements after 25 hours of interviews or focus groups
2. ulrish and eppinger
   —> 10 is not enough
   —> 50 is too many
3. virzi and Lewis
   —> 12-20 people

Question 19

What are the two main categories for data collection methods?

Answer 19

1. Question-based methods

2. Observation-based methods

Question 20

Question-based methods?

Answer 20

1. one-to-one interview
   —> in-depth interviews
2. group interviews
3. focus group interviews
4. surveys
5. Questionnaires
   —> telephone
   —> post
   —> e-mail or the web

Question 21

How should questions be structured?

Answer 21

1. Structured
   The questions are formulated and the order of the questions are defined in advance
2. Semi-structure
   —> Questions are formulated in order to guarantee that certain topics are covered but without the exact formulation and the exact order may vary between interviews
3. unstructured
   —> the interview governs the content of the interview
   —> the interviewer’s role is more passive.
   —> the questions are not formulated in advance

Question 22

How should the questions be structured in general?

Answer 22

1. Move from the general tp the more specific
   —> as a funnel
2. Move from the neutral topics the more “sensitive! topics
3. Move from questions on facts to questions which require reflection and evaluation

Question 23

Interview technique?

Answer 23

1. Probing
   —> what do you mean?
   —> why?
   —> can you demonstrate?
2. Confirming questions
   —> have I understood you correctly…?

Question 24

How should the questions be formulated?

Answer 24

1. be specific
   —> only one question at the time
2. Not be complicated
   —> do not elaborate too much in the questions
3. Be formulated in terms that could be understood without problems
   —> terms that are used by the target group
4. You should not be leading
5. Be neutral
   —> don’t be loaded in any direction (positive, negative=
6. do not include “prestige bias”
7. minimize demands on memory
8. Avoid to be hypothetical

Question 25

Describe open and closed questions

Answer 25

1. Open questions —> requires explanations —> develops the diaglog ``` 2. closed questions —> concern confirmations or denials —> have you (or not) —> is it (or is it not)? —> Can you (or can't you)? ```

Question 26

Give an example of a question put as opened and close!

Answer 26

CLOSED QUESTIONS Do you think it's difficult to book a ticket on the company's websight? OPEN QUESTION What are your thoughts about booking a ticket on the company's web sight?

Question 27

What is an indirect question?

Answer 27

1. An indirect question directs the interviewee's attention to another person (or to other persons) than the interviewee 2. Can be a way to adress more sensitive questions or areas where a true answer may not be anticipated EXAMPLE Direct: — Do you use the safety devices Indirect: — Do people working here use the safety devices?

Question 28

What factors will influence the quality of the interview?

Answer 28

``` 1. The interviewer —> social skills —> training and experience —> motivation —> safety, security ``` 2. The content —> sensitivity —> complecity —> interest. 3. The situation —> time —> place —> other people 4. The interviewee —> social skills —> ability to answer —> safety, security

Question 29

Tips on interviewing?

Answer 29

1. don't be afraid of silence 2. try not to pose leading questions 3. use probing 4. let the interviewee lead the conversation 5. do not let the interviewee lead the conversation in the wrong direction 6. do pre-tests of your interview guide 7. mediating tools are useful

Question 30

what is probing?

Answer 30

what do you mean? why? can you demonstrate?

Question 31

What is a focus group?

Answer 31

1. A group interview with a moderator 2. The aim is that the participant shall discuss with each other not with the moderator 3. 6-10 participanats 4. useful for "sensitive topics" 5. Often advocated that the Ps should be similar but... Fun facts or whatever: — Developed during WW2 - for propaganda film — forgotten until 1980

Question 32

what is the role of the moderator?

Answer 32

1. Lead and focus the discussion 2. Help the silent 3. Keep the talkative down

Question 33

Practicalities during focus group?

Answer 33

1. one moderator 2. one helper that takes notes can speed up the process 3. start with fika —> hungry people are not as engaged 4. Sandwich is good —> candy is better

Question 34

What are the relative benefits of focus of focus groups

Answer 34

1. There are no scientific proofs that focus group have benefits compared to interviews 2. There are lots of anecdotal evidence that focus groups work as advertised 3. Focus groups are fun

Question 35

What mediating tools are there?

Answer 35

Mediating tool or mediating object is here a "stimulus" 1. something which enhances reflection 2. something which enhances discussion 3. something which brings out focus 4. The purpose of using a mediating tool may be divergent or convergent

Question 36

what type of mediating tool is proper?

Answer 36

— It depends on the purpose of the study — Mediating tools may be used in user studies with the purpose to: 1. describe problems 2. elicit requirements 3. generate ideas and concepts 4. evaluate design solutions

Question 37

give examples of mediating tools!

Answer 37

1. pictures 2. video 3. collage 4. sketches 5. prototypes and mock-ups 6. "cultural probes"

Question 38

why is sketches a good mediating tool?

Answer 38

It is good to accompany with story boards

Question 39

Describe observation based methods?

Answer 39

— Observations are an efficient way to study situations — Requirements are shown ``` — As descriptor of problems: —> as comparisions —> as assumptions —> as compensating behaviour —> as actions —> as requirements —> as solutions ```

Question 40

What different types of observation are there?

Answer 40

1. Direct observation — Observation of something that takes place before our eyes. — Typically meetings, what goes on in a railway station etc. 2.Participatory observation — Common anthropological and sociological research — The observation takes part in the activities studies —> The social system that is under observation during a shorter or longer period of time 3. Self observation — A special form of self observation is when the researcher observe themselves. — e.g. diaries or photo assignments 4. natural or constructed observation 5. obtrusive or unobtrusive observation (requries acceptance and trust)

Question 41

Describe the structure during observations

Answer 41

The same spectrum as for question based methods: 1. unstructured —> trying to grasp the situation 2. Semi-structured —> having ideas what to look for —> a plan for recording and analyzing 3. Structured —> you look for specific things according to an observation protocol or similar

Question 42

Name the motives for observation studies

Answer 42

1. the customer/ user does not always want to describe problems, requirements, wishes etc. 2. It is common to blame oneself for different failures, not the design of the environment or product 3. The customer cannot always provide the information —> lack of language-skills —> compensating behaviour —> actions having become non-conscious —> regarding the requirement as self-evident

Question 43

What could be observed?

Answer 43

1. actions —> what people do as as well as what people do not —> work postures —> work sequences of action and sub-actions —> duration of actions —> ineraction patterns —> compensating behaviors 2. Communication patterns —> ""A communicates with B, while C communicates with D" —> someone looking for information (when not needed) —> someone is not looking for information (when needed) 3. Alternative usage of the product —> product is ised for different purpose than the assumed one —> used in a different manner then the assumed one 4. Modifications to the product —> in order to better fit the individual —> in order to better fit the task

Question 44

Negative thoughts about interview methods for evaluation?? and pros for observations?

Answer 44

1. Questions could disturb peoples natural way of doing things and are efter based on pre-assumptions 2.People —> people don't ask questions to problems they don't know or they don't think can be solved —> give poor results on everyday life issues, routines and habits —> often provide answered they think is expected from them 3. observation —> cannot rely on what is reported on actions, problems etc. —> by observation the customers own modifications can be identified 4. Communication is non-verbal

Question 45

Important to remember with regards to observations

Answer 45

1. to observe is not simply too look 2. to observe is a state of mind, a mission 3. The one who needs the observations done, is the one wh should carry out the observations

Question 46

Definition of context?

Answer 46

1. The part od a text or statement that surrounds a particular word or passage and determine it's meaning 2. the circumstances in which an event occurs 3. A setting 4. Context is sometimes referred to as environment 5. More than the physical environment —> circumstances in which the product will be expected to operate

Question 47

Aspects of context?

Answer 47

``` 1. Physical environment —> light —> sound —> pollutants etc. ``` 2. Social environment —> collaboration —> attitudes etc. 3. Organizational environment —> infrastructure —> support 4. Technical environment

Question 48

name some different approaches where context is stressed?

Answer 48

1. Ethnography —> writing the culture, studies with the purpose to understand, to make the implicit explicit —> participation and involvement of the researchers 2. Naturalistic studies 3. emphatic design 4. Contextual design —> contextual interview (combo of interview, observation and reconstruction of events) —> understanding develops in collaboration between user and developer —> does not impl the degree of user involvement —> more or less structures

Question 49

Name some methodical consequences?

Answer 49

1. the time that has to be spend 2. enough time to undersand the individual's objective, the motive behind an activity 3. the individual's objectives change over time 4. actions become operations over time 5. Different data collection methods and analyses are needed 6. triangulation of different data collection methods e.g. —> question based —> observation based —> earlier documentations/ research 7. Researcher's perspective —> approach and ambition will influence the outcome of the study

Question 50

what is triangulation?

Answer 50

— Different method provide different types of data, often complimentary — Qualitative data is often needed to explain quantitative data