research designs, validity, reliability and sample selection

Question 1

What is naturalistic observation?

Answer 1

• Collect data about naturally occurring behaviour
• Used when manipulation of variables is not possible/ethical/preferable
• Can still predict behaviour even though we cannot infer causes
• Good in early explanatory stages or for relating naturally occurring variables
• Third variable problem
• Direction of causality
• Solutions:
o Control for possible covariates
o Longitudinal assessment

Unique features:
o Unobtrusiveness of researcher/observer
o Researcher must remain aloof to record natural behaviour
o Researcher tries not to let participants know they are being watched
• Reactive nature of measurement and observation
o Awareness of being observed may alter behaviour
• Pro:
o Obtain accurate descriptions
• Con:
o Purely correlational, cannot isolate causes

Question 2

What is the experimental approach?

Answer 2

• Observation under controlled conditions by systematically varying one or more variables
• Used to determine cause using canons of deductive logic
o Temporal precedence: effect -> cause
o Agreement: effect when cause is present
o Difference: effect when cause absent
o Concomitant variation: rule out alternative explanations

Question 3

Outline the experimental method

Answer 3

Used to determine cause using canons of deductive logic
o Temporal precedence: effect -> cause
o Agreement: effect when cause is present
o Difference: effect when cause absent
o Concomitant variation: rule out alternative explanations
o One or more variables having at least 2 different levels that are systematically changed by experimenter
• Observe effect on DV(s)
• Control extraneous variables
• Random assignment to groups
o Evenly distributes abnormalities or irregularities across groups
• Replication necessary

Question 4

Advantages of experimental method

Answer 4

Better support for cause-effect relationships
Greater control over all aspects of research
Ability to precisely manipulate variables of interest
Findings shown to be more enduring over time (can be used as a basis for more applied research)
May need fewer PPs than non-experimental research
May be easier to write up findings

Question 5

Disadvantages of experimental method

Answer 5

Experiments are artificial and contrived
May not translate to real-life situations
May not be generalizable to other populations
May cause more reactionary response from PPs
Difficult to manipulate certain constructs

Question 6

Basic vs applied research goals

Answer 6

Basic
Advance knowledge
Identify relationships and constructs
Identify relationships among causes

Applied
Solve problems
Yield large effects
Predict future events

Question 7

Internal validity

Answer 7

Internal Validity is the approximate truth about inferences regarding cause-effect or causal relationships.
extent of cause-effect claims
o Threats:
 History: external events other than treatment between conditions
 Maturation: changes over time due to age/fatigue
 Selection: sampling bias for different conditions
 Attrition: condition-sensitive drop-out
 Testing: pretesting suggests hypothesis
 Instrumentation: change in observer measurement criteria, lack of standardisation/calibration (unreliable measures lead to low internal validity, you may get a different result every time you do the experiment)
 Statistical regression: ‘extreme’ PPs will tend to be closer to the mean on retesting
Extraneous influences on task performance.

Question 8

External validity

Answer 8

extent that findings can be generalised
o Threats:
 Selection/sampling: effect may only apply to sampled group
 Experimental setting: effects of constrained setting and PPs’ knowledge they are being tested may limit generalizability
 Testing: pre-test of other experimental treatments may result in atypical behaviour

Question 9

Construct validity

Answer 9

extent to which instruments measure what they claim to measure

Scales that have not been pre-tested. Scales should be constructed properly, possibly apply FA method to see whether all items relate to same construct.

Question 10

Convergent validity

Answer 10

Extent to which measure is consistent with other measures of the same construct

Question 11

Discriminant validity

Answer 11

 Discriminant validity: high score on construct does not equal the same as high score on measure of ‘wrong’ construct
i.e. scale should be able to discriminate between constructs. Anxiety scale should assess only anxiety, should not yield same scores as a scale measuring self-esteem as these are very different constructs.

Question 12

Reliability

Answer 12

• Comparable results over time/observations

Test should yield consistent scores for same participant over time

Question 13

Kinds of reliability for measures

Answer 13

o Test-retest reliability: test repeated over time
o Parallel-forms reliability: parallel forms of test applied over time
o Split-half reliability: parallel forms of test applied intermingled
•Observer/judgement reliability: Inter-observer reliability

Question 14

Sampling aims

Answer 14

• Population: all members of group
• Sample: group selected from population
• Aim: to get sample that is representative of target population
• Ideal: to sample proportionally, representing every type in population
• Sampling usually prone to sampling bias
• Representative sample not necessary unless the researcher needs to generalise

Question 15

Opportunity sampling

Answer 15

o Recruit whoever is available at a particular time or in a particular location
Bias- i.e. all students on the same campus

Question 16

Snowball sampling

Answer 16

 Recruit initial sample
 Initial sample recruits further
 Initial sample may pass measure on, or provide contact details for further PPs

Question 17

Volunteer sampling

Answer 17

o Requests for PPs
o Self-selection biases
o Considerable questions about generalising

Question 18

Quota sampling

Answer 18

o Designed to reflect target population
o Recruit ‘to order’
o Recruit until you acquire specified number of each specified group

Question 19

Random sampling

Answer 19

o Every member of target population has equal chance of being selected
o Often not really random

Question 20

Internet samples

Answer 20

o	Self-selected volunteers
o	Sample consists of volunteers who:
	Know how to use computers
	Have access to computers
	Know enough about the internet to find the studies
	Volunteer to participate

Question 21

Volunteer characteristics

Answer 21

• Volunteers- characteristics:

o Respondents higher on agreeableness and openness

Question 22

Volunteerism and external validity

Answer 22

o Volunteer bias may affect external validity
o 2 assumptions:
 Volunteers differ in meaningful ways from non-volunteers
 Differences between volunteers and non-volunteers affect external validity
 Question of whether they are representative of general population

Question 23

Types of research designs

Answer 23

o Experimental
o Quasi-experimental
o Correlational
o Longitudinal

Question 24

Some different methods of measurement for variabels

Answer 24

o Survey/questionnaire
popular and easy to use but may have reliability and validity issues- participants can control their responses consciously, so results may reflect demand characteristics
o Observation
Coders may bring biases into interpreting behaviour, may miss certain behaviours
o Physiological
infer link between body and mind
o Genetic
o Behavioural
may not allow determination of underlying causes of behaviour
o Implicit
validity of operationalisation of underlying processes often questioned
o Content analysis
o Interview

Question 25

Random error vs bias

Answer 25

• Random error: inconsistent and balances out, given large sample size and random assignment. should be reduced
• Bias: has a direction and must be eliminated

Question 26

Sources of bias

Answer 26

•      Participant
o      Social desirability
o	Demand characteristics
•	Researcher:
o	Experimenter expectations
o	Selective reporting of data
o	Rare occasions – falsifying data