Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

PYB210 > 3. Sampling Validity and reliability > Flashcards

3. Sampling Validity and reliability Flashcards

1
Q

what is sampling?

A

finding participants.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

why is it important to sample properly?

A

so that we can make generalizable inferences

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

population

A

a group of people about whom one would like to draw some meaningful conclusions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

sample

A

a subset of that population that is actually included in your research study

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

sample framw

A

a list of members/elements of a population from which one might obtain a sample

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

census

A

a list of all of the people comprising a particular population

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

what is critical to have when making generalisable inferences about the population on the basis of measurements of your sample?

A

representative samples

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

what are representative samples?

A

select a sample whose typical characteristic are approximately the same as the typical characteristic of the population

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

sampel statistics

A

a numeric characteristic of a sample (measured)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

population parameter

A

a numeric characteristic of the population (often not known)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Resposne rate

A

what proportion of the people responded?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

sampling error

A

the difference between the sample statistic and the population parameter (depends on the sample size)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

what is sampling bias

A

something that we would like to avoid

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

probability sampling

A

a way to ensure that your sample is representative of the population (on the characteristics deemed important for the study

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

what is the basic principle of probability sampling?

A

a sample will be representative of the population if all members of the population have an equal chance of being selected in the sample
allows the researcher to calculate the relationship between the sample statistic and the population parameter

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

what are the types of probability sample

A 
simple random sample
systematic random samle
stratified random sample
multistage cluster sampling
Multi-stage / multi-phase sampling
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

simple random sampling

A

each member has an equal and independent chance of being selected.
you define the population - list all members - assign numbers.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

how would you do random sampling>

A

using a table of random numbers to select. Use a lottery method
use a computer program to randomly select

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

systematic random sample

A

every kth person (k=number).
randomly selected the first person then divides the size of the population bu the size of the desired sample, and use this to determine the interval at which sample is selected.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

what must you ensure when doing a systemtic random sample?

A

that the list of elements is not arranged in a way that means systematic sampling could lead to biased sample (e.g. GPA order)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

stratified samplign

A

if you want to make sure the profile of the sample matches the profile of the population on some important characteristic.
Researcher divides the population into sub populations (strata) and randomly samples from the stata

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

why use stratified sampling

A

can reduce sampling error by ensuring ratios reflect actual population (e.g., ratio of males to females)
To ensure that small subpopulations rations reflect actual population (e.g. ratio or males and females)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

multi-stange cluster samplingg

A

• Begin with a sample of groupings and then sample individuals
• E.g. Rural sample
o Define rural townships as those with populations

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

when might you use multi-stage cluster sampling?

A

• hunger games – different districts, different characteristics,

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Multi-stage / multi-phase sampling

A
  • Larger sample obtained first in order to identify members of a sub-sample
  • Sub-sample then randomly chosen from for study
  • Good (but costly) way to identify not readily identifiable subgroups
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

advantages of probability sampling?

A

helps overcome sampling bias

- representativeness

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

disadvantages with probaility sampling?

A

It’s all very well selecting people at random, but the fact that you have selected them doesn’t mean that they will take part in your study…

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

non-probability sampling

A

Not every member of the population has an eqyal chance of being part of the sample

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

why use non-probability sampling?

A

o There are no lists for some populations under study, e.g.
• The homeless
• Certain occupations (e.g., farmers)
• Hidden populations (e.g., people involved in “clandestine” activities)
• Convenience/ resource restrictions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

convenience samples

A

• A sample of available participants, e.g.,
o students enrolled in a particular course
o People passing a particular location

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

advantages of convenience samples

A

easy cheap

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

disadvantages of convenience samples

A

no control over representativeness

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

snowball sampling

A

Involves collecting data with members of the population that can be located and then asks those members to provide information/contacts for other members of the population

Used mainly for hard to study populations, e.g.,
o Gay men
o Homeless young people
o Illegal immigrants

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

Quota sampling

A
  • Non-probability sampling equivalent of a stratified random sample
  • Want to reflect relative proportions of a population
  • But you don’t/aren’t able to sample randomly from each strata as you do in stratified random samples
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

Purposive / judgement sampling

A

Clear purpose to the sampling strategy: select key informants, atypical cases, deviant cases or a diversity of cases.

36
Q

why is Purposive / judgement sampling used

A

o Select cases that might be especially informative
o Select cases in a difficult-to-reach population
o Select cases for in-depth investigation

37
Q

what to consider when selecting a method of sampling?

A
  • As a major aim of quantitative research is the ability to generalise results the ultimate method is probability sampling one
  • However this is often not workable or feasible given resources, time, the specific target population
  • Sampling method used should be fully explained and caveats about the likely generalisability of results made accordingly so that the reader can review your results in an informed way
38
Q

sample size

A

as we have already seen the size of your sample can influence how representative it is of the population
important that you have an appropriate size

39
Q

how many participants do i need for the study?

A

o Largely determined by the analysis you plan to conduct with the data derived
o Generally the more complex the analysis the larger the sample you require
o Increases in sample size bring with them increases in accuracy/precision/reduces sampling error

40
Q

what does heterogenity of the population mean?

A

greater variation in the population, the larger the sample should be

41
Q

when is a larget sample size needed?

A

o heterogeneous
• composed of widely different kinds of people
o you want to breakdown the sample into multiple subcategories
• e.g., look at males and females separately
o when you expect a small effect or weak relationship
o when you use less efficient methods of sampling
• e.g., cluster sampling
o for some statistical techniques

42
Q

what are the five simple rules for determining sample size?

A
  1. if population is less than 100, use entire population
  2. larger sample sizes make it easier to detect an effect or relationship in the population
  3. compare to other research studies in area by doing a literature review
  4. use a power Table for a rough estimate
  5. use a sample size calculator (e.g., G-Power)
43
Q

What does it mean to operationalise the IVs?

A

How are you going to manipulate it? How is it manipulated (if you cant)?

44
Q

what does it mean to operationalise the DVs?

A

how are you going to measure it?

45
Q

what does reliability question?

A

does our measurement instrument behave sensibly? Does it always measure the same thing in the same way?

46
Q

what does validity question?

A

are we measuring wht we think we are measuring?

is our measure credible, is it believable?

47
Q

How do we asses whether our measures/operationalisations are good?

A

we test validity and reliability

48
Q

when can you test reliability and validity?

A

not until after the the questionairre is developed and used

49
Q

why is pilot test beneficial?

A

so that you can determine whether the means of measures are reliable and valid.

50
Q

what is relaibility?

A

the consistency or repeatability of your measure

51
Q

can a measure be reliabile but not valid?

A

you could have consistent measure that does not accurately measure the construct

52
Q

can a measure be valid but not reliable?

A

NO! if your measure doesnt consistently and dependably measure the construct it cannot possibly be measuring what it says its measuring

53
Q

what are types of reliability?

A

stability of the measure (tes-retest)
Internal consistency of the measures (split-half, crohbach’s alpha)
Agreement or consistency across raters (inter-rater)

54
Q

what does test-retest reliability question?

A

does your test measure the same thing every time yo uuse it?

55
Q

how do you perform a test-retest test?

A

address the stability of your measure
you administer the measure at one point in time (time 1)
you then give the same measure to the same participants at a later point in time (time 2)
You correlate the scores on the two measures

56
Q

what are the two main problems with test-retest

A

memory effect, practice effect

57
Q

memory effect

A

you might remember the question and look up the ones you didnt know, therefore, your performance might get better from time 1 to time 2

58
Q

practice effect

A

performance improves because of practice in taking the test

59
Q

what are other considerations to make when performing a test-retest test?

A

intervals

60
Q

what is something to consider when thinking about intervals for a test-retest?

A

if too short theres as greater risk of memory effects

if too long theres a risk of other variables (e.g. additional learning) that may influence the result

61
Q

What does the split-half reliability question?

A

is your measure internally consistent?

62
Q

How does one perform a split-half reliability test?

A

you administer a single measure at one time to a group of participants
But, for your purposes you split the measure into two halves and you correlate the scores on the two halves of the measure (higher correlation means greater reliability)

63
Q

what are the strengths of split-half reliability?

A

eliminates memory & practice effects

64
Q

what are the limitations of the split-half reliability?

A

the two halves may not be completely equivalent

65
Q

What does Cronbach’s alpha assess?

A

the internal consistency of your measure. i.e. tells you how well the items or questions in your measure appear to reflect the same underlying construct

66
Q

what is the range of coefficient for Cronbach’s alpha?

A

0 - 1.00

the closer it is to 1 the better the reliability of the measure

67
Q

what does inter-rather or inter-observer reliability question?

A

Do different raters measure the same thing?
It checks the match between two or more raters or judges.
e.g. research investigating the relationships between communication and family functioning

68
Q

how does one calculate inter-rater reliability?

A

Nominal or ordinal scale - the percentage of times different raters agree
Interval or ration scale - correlation coefficient

69
Q

what kind of reliability co-efficients should i be aiming for test-retest coefficients?

A

below 0.7

70
Q

what kind of reliability co-efficients should i be aiming for Internal consistency?

A

below 0.7

71
Q

what kind of reliability co-efficients should i be aiming for rating consistency

A

below 0.9

72
Q

why is validity an issue?

A

many (if not most) variables in social research cannot be directly observed
e.g. motivation, satisfaction, helplessness

73
Q

what are types of validity?

A

face, content, criterion (concurrent, predictive), construct (convergent, discriminant / divergent)

74
Q

what does face validity question?

A

does it (subjectively) look like it measures what we want it to measure?

e. g. on the face of it, which of the following is a more valid measure of worker morale?
- no of grievances filed with the union
- no of books borrowed by workers during off-duty hours

75
Q

what is the problem of face validity?

A

it is a weak subjective method for assessing validity but it is a first step.

76
Q

content validity

A

the extent to which the measure represents a balanced adequate sampling of relevant dimensions of the measured construct

77
Q

what does content validity question

A

does it cover all aspects of the construct that it purports to measure?

78
Q

Criterion-related validity

A

involves checking the performance of your measure against some external criterion

79
Q

what are the two types of criterion-related validity?

A

concurrent and predictive

80
Q

what does concurrent, criterion-related validity question?

A

does it relate to a known criterion, for example, an alternative (gold standard) measure of the same construct?

81
Q

what does predictive, criterion-related validity question?

A

does the measure predict/relate to some criterion that you would expect it to predict?
Does the measure differentiate between people in the way you would expect ?
What should measures of the following constructs predict?

82
Q

Criterion-related validity: concurrent validity

A

Establish the validity of your measure by comparing it to a “gold standard” (i.e., existing validated measure of the same construct)

83
Q

In concurrent validity, if theres already an existing, validated measure, why is it necessary to come up with a new one?

A

Your test might have some time/cost savings, e.g., shorter, easier to administer
Conversely your test might aim to demonstrate a more nuanced understanding of the construct in question (and be longer/more complex)

84
Q

Construct validity

A

Establishes validity by showing that your measure relates to other constructs in a way that you would expect (theoretically)
e.g., you would expect a measure of marital satisfaction to be positively related to respect for partner and negatively related to marital infidelity

85
Q

what are the two types of construct validity?

A

convergent

divergent

86
Q

convergent validity

A

Measures of constructs that theoretically should be related to each other, are, in fact, observed to relate to each other (i.e., there is correspondence or convergence between similar constructs)

87
Q

divergent validity

A

Measures of constructs that theoretically should not be related to each other, are, in fact, observed not to relate to each other

PYB210 (11 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions