Experimental Designs

Question 1

What is the independent variable ?
I predict that increasing recommended offers in Amazon will result in increasing products sales.

Answer 1

Recommended offers.

Question 2

What are the types of research ?

Answer 2

Descriptive (y) , Relational (x <—> y), Experimental(x -> y)

Question 3

What type of research :
Describes a situation or a set of events.
X happens.

Answer 3

Descriptive

Question 4

What type of research :
Identifies relation between multiple variables.
X is related to Y.

Answer 4

Relational

Question 5

What type of research :
Identifies causes of a situation or a set of events.
X is responsible for Y.

Answer 5

Experimentalcontextual setting (enternal validity, you can apply it or based in the real world / less control)

Question 6

Descriptive Research uses what types of methods ?

Answer 6

Qualitative Methods (e.g., interviews)
Quantitative Methods (e.g., usage analytics)

Question 7

______________ assignment means assigning individuals at random to groups in an experiment, so that each individual of the population has the same chance of becoming a participant in the study

Answer 7

Randomizing

Question 8

The non-random arrangement of experimental units into groups (blocks/lots).____________ reduces known but irrelevant sources of variation between units and thus allows greater precision in the estimation of the source of variation under study.

Answer 8

Blocking

Question 9

_____________It is an inductive estimate of the degree to which conclusions about causal relationships can be made, based on the measures used, the research setting, and the whole research design

Answer 9

Internal Validity in a controlled setting (higher internal validity , you can be sure x is causing y , you have full control)

Question 10

______________It concerns the extent to which the (internally valid) results of a study can be held to be true for other cases, for example to different people, places or times. In other words, it is about whether findings can be validly generalized

Answer 10

External Validity in a contextual setting (higher external validity, you can apply it or based in the real world and you have less control)

Question 11

Usability is described as the extent to which a product can be used by specified users to achieve specified goals with __________________________ in a specified context of use

Answer 11

effectiveness, efficiency, and satisfaction

Question 12

Effectiveness is

Answer 12

accuracy and completeness with which users achieve specified goals

Question 13

Efficiency is

Answer 13

resources used in relation to the results achieved Typical resources include time, human effort, costs, and materials.

Question 14

Satisfaction

Answer 14

the extent to which the user’s physical, cognitive and emotional responses that result from the use of a system, product, or service meet the user’s needs and expectations

Question 15

____________ are when the chef tastes the cake
finding and fixing usability problems. You can quantify the problems in terms of frequency and severity, track which users encountered which problems, measure how long it took them to complete tasks, etc.

Answer 15

Formative Tests

Question 16

___________ are when the guests taste the cake
describing the usability of an application using metrics. This can be, for instance, a
comparison of different versions of a product or comparison of competing products.

Answer 16

Summative Tests

Question 17

What are the types of evaluation:

Answer 17

1.Controlled settings involving users
Users’ activities are controlled in order to test hypotheses and measure or observe certain
behaviors. The main methods are usability testing and experiments.
2. Natural settings involving users
There is little or no control of users’ activities in order to determine how the product would be
used in the real world. The main method used is field studies.
3. Any settings not involving users
Consultants and researchers critique, predict, and model aspects of the interface in order to
identify the most obvious usability problems.

Question 18

Qualitative research helps us understand:

Answer 18

• Behaviors, attitudes, and aptitudes of potential and existing product users
• Technical, business, and environmental contexts - the domain - of the product to be designed
• Vocabulary and other social aspects of the domain in question * How existing products are used

Question 19

Give controlled or contextual user environment examples :

Answer 19

Controlled environment - Lab usability tests, Focus groups , Participatory design Interviews
Medium contextual - Remote usability test,Intercept study , Interviews
Highly contextual - Field Studies, Diary Studies, Interviews

Question 20

__________ can be extremely flexible and vary between Unstructured, Semi-Structured, and Structured
Structured, closed questions limit users to a small number of predefined choices. Examples include yes-no questions, multiple choice, true-false, and Likert-scale questions, asking for ratings on a scale of 3, 5, 7, or more possibilities

Answer 20

Interviews

Question 21

___________ reliability refers to the extent to which two or more individuals agree.

Answer 21

Interrater

Question 22

A mix of direct observation and user interviews to better understand context-of-use
__________________ is based on a master-apprentice model of learning: observing and asking questions of the user as if she is the master craftsman, and the interviewer the new apprentice

Answer 22

Contextual inquiry

Question 23

___________ is a visualization that tells a narrative of the end user trying to accomplish a specific task or goal
Persona development is focusing on end-users of the software being developed, and Narrative Journey Maps capture their interactions and help define goals/motivations and pain points.

Answer 23

Journey Map

Question 24

____________ is a technique to understand how users organize information and concepts into categories. It can be a valuable tool to uncover one aspect of a user’s mental model, the technique assumes that the subject has refined organizational skills and that how they sort a group of abstract topics will correlate to how they will end up wanting to use your product.
It can also be moderated or unmoderated.

Answer 24

Card Sorting

Question 25

__________ are quite useful for gauging initial reactions to a product’s form.It can also gather reactions to a product that the respondents have been using for some time.

Answer 25

Focus groups

Question 26

_____________ is an expert review method in which interface experts simulate users, “walking through” a series of tasks,
Because of the exploratory nature of a cognitive walkthrough, it can give an understanding of how users might interact with an interface the first time that they attempt to use it

Answer 26

A cognitive walkthrough

Question 27

In a___________, an expert takes a set of heuristics (rules of thumb) and compares the heuristics to the interface in question
They are short sets of usually no more than 10 interface rules. To be truly effective, the expert must be very familiar with the heuristics and have previous experience in interpreting them.

Answer 27

Heuristic review

Question 28

_________ is important when choosing a dependent variable. A measure is perfectly ______ if you get the same result every time
you repeat the measurement under identical conditions.

Answer 28

Reliability, Reliable

Question 29

What is Task completion time ?

Answer 29

Time from users who completed the task successfully

Question 30

______________ is a scientific discipline concerned with the construction of assessment tools, measurement instruments, and formalized models that may serve to connect observable phenomena (e.g., responses to items in an IQ-test) to theoretical attributes (e.g., intelligence)

Answer 30

Psychometrics

Question 31

______________ is a body of related psychometric theory that predicts outcomes of psychological testing such as the difficulty of items or the ability of test-takers
It is a theory of testing based on the idea that an observed or obtained score (X) on a test is the sum of a true score (T) and an error score (E)

Answer 31

Classical Test Theory (CTT)

Question 32

Consistency of results when you repeat the same test on the same sample at a different point in time. Use Pearson or Intraclass Correlation Coefficient (ICC)

Answer 32

Test-retest Reliability

Question 33

The extent to which two or more raters (or observers, coders, examiners) agree. Different from Intra-rater reliability (i.e., agreement among repeated administrations of a test by a single rater). It can also use ICC, percentage agreement, or kappa tests.

Answer 33

Inter-rater Reliability

Question 34

_________ can be inferred by testing the internal consistency of items. For that, researchers use a measure of internal consistency known as Cronbach’s α

Answer 34

Reliability

Question 35

_____________ is a paradigm for the design, analysis, and scoring of tests, questionnaires, and similar instruments measuring abilities, attitudes, or other variables
IRT models are often referred to as latent trait models
The term latent is used to emphasize that discrete item responses are taken to be observable manifestations of hypothesized traits, constructs, or attributes, not directly observed, but which must be inferred from the manifest responses.

Answer 35

Item Response Theory

Question 36

__________ is the extent to which a test accurately measures what it is supposed to measure
It refers to the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests

Answer 36

Test validity

Question 37

Examples of post-study questionnaires:

Answer 37

• SUS - System Usability Scale (10-item)
• QUIS - User Interface Satisfaction (6 to 27 items)
• SUMI - Software Usability Measurement Inventory (50-item) * PSSUQ - Post-Study Usability Questionnaire (13 to 19 items) * UMUX - Usability Metric for User Experience (4-item)
• UMUX-LITE (2-item)

Question 38

___________ is shown to provide a reliable measure for ease of use when applied after each task

Answer 38

Single Ease Question (SEQ)

Question 39

The scales of the _________________________cover both classical usability aspects, also known as pragmatic usability (efficiency, perspicuity, dependability), and user experience aspects, also known as hedonic usability (novelty, stimulation, attractiveness)

Answer 39

User Experience Questionnaire (UEQ)