Selecting Measures and Non-Experimental Methods I: Observational and Survey Research

Question 1

every measure we obtain consists of:

Answer 1

“true score” and error

Question 2

error is due to:

Answer 2

1. bias (a systematic deviation that is the result of confounds)
2. random error (a result of nuisance)

Question 3

what are three sources of measurement error?

Answer 3

1. experimenter
2. participant
3. observer/scorer

Question 4

experimenter error examples?

Answer 4

random error, and bias error (experimenter characteristics and expectancies)

Question 5

solutions for experimenter errors?

Answer 5

standardize testing conditions, standardize appearance of experimenter/replicate experiment with different experimenters, standardize coding schemes/automated recording equipment/single blind research

Question 6

examples of participant error

Answer 6

carelessness and distraction (contributes to nuisance) and participant bias

Question 7

solutions for participant errors

Answer 7

set clear task instructions with emphasis on accuracy, include a manipulation check

Question 8

what are some causes for participant bias

Answer 8

Demand characteristics and good participant effect together can cause a “pact of ignorance” - invalidates results

Question 9

demand characteristics

Answer 9

features of an experimenter that seem to inadvertently cause participants to act in a particular way

Question 10

good participant effect

Answer 10

tendency for participants to behave as they perceive the researcher wants them to behave

Question 11

how to control for demand characteristics

Answer 11

conduct double-blind research (removes confounds, but not nuisance), can use deception but this could inadvertently cause demand characteristics

Question 12

response set

Answer 12

when the context affects the way a participant responds, can be a factor of the experimental setting or the questions that are asked (social desirability could influence answers)

Question 13

response set contributes to:

Answer 13

response bias (participant bias)

Question 14

to control for yea/nay-sayers:

Answer 14

include both agree and disagree terms with switched implications, randomize question presentation (reverse-coding), care review of response set, use of pilot tests

Question 15

observer error is only present in:

Answer 15

behavioural studies

Question 16

examples of observer error

Answer 16

random observer error, observer/scorer bias-confirmatory bias

Question 17

how to control for observer error:

Answer 17

eliminate human observer (muse mechanical measures), limit observer subjectivity (standardized coding schemes), make observer “blind”

Question 18

construct validity

Answer 18

the extent to which the manipulation or measure actually represents the claimed construct

Question 19

1. establishing reliability (criteria for validity)

Answer 19

test-retest reliability, inter-rater reliability, internal consistency (research finding must be repeatable and consistent)

Question 20

what is internal consistency

Answer 20

extent to which responses to items that propose to measure the same unidimensional construct are similar, test by Split-half correlation, Cronbach’s Alpha, improved by adding items/questions

Question 21

2. content validity (criteria for validity)

Answer 21

extent to which a measure covers all aspects of a construct

Question 22

3. convergent validity (criteria for validity)

Answer 22

extent to which a measure correlates with other indicators of the same construct

Question 23

4. discriminant validity (criteria for validity)

Answer 23

extent to which your measure is distinguishable from other constructs (both related and unrelated)

Question 24

sensitivity

Answer 24

ability of measures to DETECT effects

Question 25

how to achieve sensitivity in measurement?

Answer 25

use measures with score variance (avoid all or nothing, add scale points, use pilot test) and avoid restriction of range (to avoid floor/ceiling effects)

Question 26

what is the main identifying point for descriptive research methods

Answer 26

cannot infer causation

Question 27

what are types of descriptive studies?

Answer 27

(archival), case studies, naturalistic observation, participant observation, clinical perspective

Question 28

archival studies

Answer 28

using data that has been previously recorded to answer a new question

Question 29

case study

Answer 29

collecting detailed info about the behaviour of a single person (used to study one person over an extended period of time, in depth examination is qualitative)

Question 30

con’s of case study

Answer 30

generalization based on N of 1, precludes cause and effect

Question 31

naturalistic observation

Answer 31

observing behaviour in the real world unobtrusively

Question 32

participant observation

Answer 32

when the observer imbeds themselves within the group being studied

Question 33

clinical perspective

Answer 33

descriptive approach aimed at understanding and correcting a particular behavioural problem

Question 34

how is clinical perspective different from participant observation?

Answer 34

1. the client chooses the clinician
2. clinicians cannot be unobtrusive/passive, they have been asked to participate in the situation
3. participant observer’s goal is understanding, whereas the clinician’s goal is helping

Question 35

reactivity

Answer 35

when knowledge one is being observed affects his/her behaviour

Question 36

observational techniques are high on:

Answer 36

external validity (can generalize easily to study) but low on internal validity (cannot be sure DV is caused by “IV” - because of lower control of study)

Question 37

what are some more issues with observational techniques?

Answer 37

low on objectivity, cannot make cause/effect statement

Question 38

descriptive surveys

Answer 38

seeks to determine what % of the population have particular characteristics, beliefs, or behaviours. Where one question tests for one quality. The goal is to get it to be as representative of the population as possible.

Question 39

Self-selection bias

Answer 39

those who have experienced something they were unhappy with are more likely to respond

Question 40

sampling and self-selection bias in surveys/questionnaires decrease:

Answer 40

representativeness of the sample and thus compromises generalizability

Question 41

analytic surveys

Answer 41

seeks to determine the relevant variables and how they are related, establishes correlation ranging from -1 to 1 (magnitude of 1 = strength of relationship)

Question 42

surveys/questionnaires tend to examine:

Answer 42

an opinion or belief

Question 43

tests/inventories are more ….. compared to surveys/questionnaires

Answer 43

objective, it is a specific assessment of trait, quality or scale.
ex. achievement tests (BAR, candidacy) or aptitude tests (MCAT, SAT, LSAT)

Question 44

single strata approach

Answer 44

select from one subgroup of a population

Question 45

cross-sectional approach

Answer 45

compares multiple subgroups at the same time

Question 46

longitudinal research

Answer 46

looks at one group over an extended period of time

Question 47

the group in a longitudinal study is called a:

Answer 47

cohort (may display certain traits that makes generalization more difficult)