Research Methods HFE Flashcards
Quest for fundamental Understanding? YES. Considerations of use? YES
User-inspired basic research
Quest for fundamental Understanding? NO. Considerations of use? YES
Pure applied research
Quest for fundamental Understanding YES. Considerations of use? NO
Pure BASIC Research
Foundations of Science
- Based on Empiricism - knowledge by observation
- Self-Correcting- Continually test statements with observations and revise scientific statements
- Systematically apply the scientific method
Scientific Method
LOGICAL APPROACH to obtaining answers to questions
Steps of Scientific Method
Observe, State Problem, Develop Hypothesis, Conduct Experiment, Evaluate Hypothesis, Disseminate Information
3 Goals of Science
- Description
- Prediction
- Explanation/Understanding
How are the 3 goals of science achieved?
Collecting Data (gathering facts) to build a theory. The theory provides the best explanation for the findings in the research.
What are 4 benefits of THEORY?
- Enables sensible interpolation to a specific real-world problem when there are no data.
- Provides quantitative predictions of the type demanded by engineers and designers.
- Allows the practitioner to recognize relations between problems that seem unrelated on the surface.
- Can be used cheaply and effectively to aid system design.
How do we understand the problem and how do we know what to measure?
Defines a domain of interest
In HFE, this is usually human performance within a system
Defines the conditions under which we can make useful measurements
If we are interested in the effects of cell phone use while driving, then we need to see people using cell phones while driving
What are variables?
Since conditions can vary we have variables
They are anything that can vary or differ
Can be an event, situation, behavior, or individual characteristic
We have to define our variables before we begin our research
What is an operational definition of a variable?
A definition of the variable in terms of the operations or techniques the researcher uses in order to measure or manipulate it
Why define variables this way?
Forces us to describe abstract concepts in concrete terms
Example 1: What type of patients with Type 2 Diabetes are more likely to engage in diabetes self-care?
We define “diabetes self-care” as number of times using a glucometer per week divided by the weekly number recommended by their doctor, plus the percent of recommended eye care visits they attend each year
How are variables classified?
Independent
Dependent
Subject
What is an independent variable?
The variables that are manipulated or chosen by the researcher
Example 1: You are interested in how stress affects terrain learning in a military training exercise. You directly manipulate stress in your human subjects and measure how it affects learning.
Example 2: You are interested in how genotype affects terrain learning in a military training exercise. You select participants with genotype A and genotype B and measure how it affects learning.
What is a dependent variable?
The variables that are measured by the researcher (in other words, the variables you want to know). They “depend” on the independent variable because you set up your experiment with the prediction that the dependent variable(s) will change based on the independent variable(s)
What is a subject variable?
Human characteristics AKA individual differences
Ex. Height, weight, gender, age, physical disability, personality, etc.
What is meant by the term Reliability?
The consistency of measurements
For example, if you give a test to the same group of people at two different times, the test is said to have high “test-retest” reliability if the scores for each person are similar for the two administrations of the test.
Any measure has 2 parts, what are they?
True effects
Random error
What is meant by the term VALIDITY?
The degree to which an experiment, a procedure, or a measurement represents what it is supposed to represent
What is construct validity?
does the measure that is employed actually measure the construct it is intended to measure?
What is Internal validity?
can the relations observed can be attributed with a high degree of confidence to the variables of interest? i.e., the ability to draw conclusions about causal relationships from our data.
What term describes this: : can the results or the principles derived from the results can be generalized to a variety of other settings?
External validity
What term describes this: do the behaviors observed in the study reflect the behaviors that actually occur in a natural setting?
Ecological validity
What types of Research Methods are used in Human Factors?
Descriptive
Correlational and Differential
Experimental
What is Descriptive Method?
When you want to examine a situation that cannot be replicated
You are unable to exercise any control over the events under the investigation
Control can lead to a loss of ecological and external validity
Using sources like Archival Data to obtain information about a system
Archival data is data obtained from preexisting data collections like the census
These are all describing what Method?
Naturalistic observations and ethnographic methods
Great ecological validity
Used to characterize how people interact with a system in the “real world”
Measurement issues:
Observations can be recorded at the time they are made or later
Content and amount of detail in observations vary
Length of time during which observations are made can be short or long
Observations can vary in terms of the amount of inference, or degree of interpretation, that is required to classify events into measurement categories
DESCRIPTIVE
What is the best way to begin addressing a problem for descriptive method?
Surveys and Questionnaires
The best way to begin addressing a problem by asking people in the natural environment
Steps to a good survey/questionnaire
Decide on the information you want the questionnaire to provide
Decide what type of questionnaire should be used
If you decide to write your own questionnaire, the third step is to write a first draft
Revise questionnaire
Pretest the questionnaire
Finalize the questionnaire
In Which Research Method would Interviews and Focus Groups’ work well for?
Descriptive BECAUSE Structured and unstructured interviews with users at any phase of the research process and for a variety of purposes
Do not work well for discovering specific usability problems in a system
Must decide ahead of time which behavioral variables we will measure
Measure the degree of relationship between the variables
This can be used with any data obtained from Descriptive Methods
Valuable because they enable prediction based on the previously observed relationships
Correlational and Differential Research
What is Correlational research?
measure the strength of a relationship between two or more variables
you cannot determine causality with this method
What is Differential research?
observe two or more groups that are differentiated on the basis of some preexisting variable
you cannot determine causality with this method
3 Defining Features Experimental?
Test a hypothesis that makes a causal statement about the relation among variables
Compare a dependent measure at at least two levels of an independent variable
Randomly assign people to experimental conditions to make sure that the effects of many potentially confounding factors are distributed equally across conditions
2 BASIC DESIGNS of EXPERIMENTAL METHOD?
Between subjects
Within subjects
When Experimental method is between subjects…?
Two or more groups of people are tested and each group received only one of the treatment conditions of the independent variable
Typically random assignment
Matched designs are better
When Experimental Method is WITHIN SUBJECTS….then…?
Use the same participants in each condition
Increases sensitivity of the design (more likely to find significant effects)
Reduces the number of participants needed
Major drawbacks of Within Subjects for Experimental Method?
Carryover effect: previously received treatment conditions influence a subject’s performance on subsequent conditions
Practice/Fatigue effects: increase/decrease in performance due to experience
Whenever possible USE EXPERIMENTAL DESIGNS BECAUSE…?
Optimizes internal validity
With the right setting, can be excellent external validity
Descriptive methods can provide info about real-world systems that cannot be obtained from controlled experiments
Good for obtaining quick info about user characteristics and usability
Descriptive methods can provide info about real-world systems that cannot be obtained from controlled experiments
Good for obtaining quick info about user characteristics and usability
What 6 Things does Design Involve?
Designing new products
Modifying existing products
Designing environments
Safety
Develop training programs or instruction manuals
Organizational development and restructuring
What must be kept in mind when working on design?
USABILITY!!
What is meant by the term USABILITY? (5 things)
Learnability” – how easy is it to learn?
Efficiency – how quickly and accurately can you do it?
“Memorability” – how easy is it to remember?
Low Error Rate – reduce the number of possible errors
Satisfaction – increase the user’s satisfaction with the system
Who is Don Norman?
Author of The Psychology of Everyday Things
Defined 7 principles for transforming difficult tasks into simple ones:
What are the 7 principles for transforming difficult tasks into simple ones? (Norman)
Knowledge in the World + Head Simplify the Structure of Tasks Make Things Visible Get the Mapping Right Exploit the Power of Constraints Design for Error Standardize
What is meant by Knowledge in the world?
Sources of information outside of the individual
Ex. Maps, signposts, labels, recipes, speedometer in a car
What is meant by Knowledge in the HEAD?
Information from the world that has been internalized
Ex. Affordances
How do we Simplify the Structure of Tasks?
Provide simple mental aids
Ex. Warning labels
Manage visible information
Ex. Provide speedometers to drivers
Change the nature of the task
Ex. Cashiers scan barcodes instead of typing in numbers
Making a task too simple can cause the user to lose some control of the system
Ex. Preprogrammed radio buttons inhibit finding new stations
How do we make things VISIBLE?
Feedback
When a user interacts with a system, the system should provide information that the interaction was successful or an error was made
(diagram of human machine system)
What is Gulf of Execution?
The gap between the user’s goal of action and the means to execute that goal
What is The degree to which the system provides representations that can be directly perceived and interpreted in terms of the expectations and intentions of the user?
Gulf of Evaluation
Get the Mapping Right
“Make sure that the user can determine the relationships:
Between intentions and possible actions
Between actions and the effects on the system
Between actual system state and what is perceivable by sight, sound, or feel
Between the perceived system state and the needs, intentions, and expectations of the user
“Response Compatibility”
Movement of control should match outcome goal
When you turn the steering wheel left, the car should turn left
Constraints can be used to
prevent error (Affordances, Only one option, forcing function)
Design for Error!
People will make errors
Make it easy to recover from them!
Design around them when possible
When you can’t exploit the previous principles…
STANDARDIZE: make everything the same
Creates Knowledge in the Head
Ex. Driving on the right side of the road
Without conforming to standards,
you can confuse people which causes errors
Ex. Walking on the left side of a two-way door
What are some quantitative benefits to using HFE Principles?
Increased sales Decreased training costs Decreased customer support costs Decreased development costs Decreased maintenance costs Number of chronic injuries Medical and rehabilitation expenses Number of citations or fines Decreased user errors Increased productivity Improved quality of service Decreased user turnover Increased employee satisfaction Decreases in sick leave Decreased number of accidents Number of lawsuits
What are some quantitative benefits to using HFE Principles?
Increased sales Decreased training costs Decreased customer support costs Decreased development costs Decreased maintenance costs Number of chronic injuries Medical and rehabilitation expenses Number of citations or fines
What is Front End Analysis?
Before any design solutions are generated, answer the following:
Who are the system users?
What are the major functions of the system?
What are the environmental conditions?
What are user preferences and requirements?
User Analysis
Who are your users?
Characteristics of users
Include all types of current users
Anticipate for potential users
Develop personas
Hypothetical person derived from real user statistics
Function Analysis
What functions should be performed by the system?
Functions represent a general goal, not the actual tasks
Ex.
Function: Record a TV show
Tasks: Turn on the TV and cable box, select the channel to record, select the time period to record, set recording settings
Steps to Task Analysis?
Define the Purpose
Collect Task Data
Summarize Data
Analyze Data
Steps to Task Analysis?
Define the Purpose
How will the data be used?
How are processes related? (e.g., Information flow, Task sequence)
What type of data should we collect?
Collect Task Data
Under what conditions will this be performed?
Collection methods: observation, think-aloud, task performance with questions, interviews, and focus groups
Summarize Data
Identify the task process by using an outline, flow chart, timeline, or map
Analyze Data
Determine the quality of a task by conducting a workload analysis, simulation and modeling, safety/hazard analysis, or scenario specification
Heuristic Evaluation
Judge the compliance with HFE principles
Should be done by several people to compare results
Not very scientific
Good to get started
Have users interact with a system to identify flaws Typically a functional product Conducted by: Giving the user a scenario and a task to complete Videotape Think aloud protocol Answer questions Debrief/ questionnaire Data Collected: Time to complete task Errors Comments
Usability Testing
Final Test and Evaluation
Based on what you found…
Repeat the process in a more directed manner
Reach an evaluation decision
