Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Pyb210 - Statistics > WEEK 2 - Measuring variables, sampling, validity and reliability > Flashcards

WEEK 2 - Measuring variables, sampling, validity and reliability Flashcards

1
Q

Why is generalisability in sampling important?

A

Generalisable results are results that reflect the true state of affairs in the population of interest - to claim this your sample needs to be as representative of the population as you can make it

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is ‘population’

A

The totality to whom/which you wish to generalise your study findings

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is ‘sample’

A

the participants in your study

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What are the two types of sampling procedures?

A

Probability sampling - simple random, systematic random, stratified, multi-stage cluster

Non-probability sampling - convenience, snowball, purposive

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is probability sampling?

A
  • A way to ensure that your sample is representative of the population (on the characteristics deemed important to the study)
  • Basic principle: A sample will be representative of the population if all members of the population have an equal chance of being selected for the sample
  • Allows the researcher to calculate the relationship between the sample and the population
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What are the types of probability sample?

A
  • Simple Random sample
  • Systematic random sample
  • Stratified random sampling
  • Multi-stage cluster sampling
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What is a simple random sample?

A
  • each member has an equal and independent chance of being selected
  • define the population, list all members, assign numbers then randomly select number
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What is systematic random sampling?

A
  • Every kth person

-Randomly select the first person then divide the size of the population by the size of the desired sample and then use this to determine the interval at which the sample is selected

**size of population/size of desired sample **

Example: to select a sample of 1000 people from a list of 10 000 randomly select the first person and then select every 10th person from the list

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is stratified sampling?

A
  • If you want to make sure the profile of the sample matches the profile of the population on some important characteristics (for example, age or ethnicity)
  • Researcher divides population into sub-populations (strata) and then randomly samples from strata
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Why do we use stratified sampling?

A
  • Can reduce sampling error by ensuring ratios reflect actual populations (example -ratio of different ethnic groups)
  • To ensure that small sub-populations are included in the sample
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is multi-stage cluster sampling

A
  • Begins with a sample of groupings then samples individuals

example: Rural sample
- Define rural sample as those with populations <X
- Get a listing of all relevant towns
- Take a random sample of towns
- Randomly sample people from within the randomly sampled town

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is the difference between multi-stage cluster sampling and stratified sampling?

A

Multi-stage cluster is not the same as stratified sampling as each cluster does not need to be sampled.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is multi-stage / multi-phase sampling?

A
  • Larger sample obtained first to identify members of a sub-sample
  • Sub-sample randomly chosen from study

Example: Large community survey in Australia which asks if they had diagnosis X disease –> X disease sufferers followed up again for sampling

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is non-probability sampling?

A
  • Not every member of the population has equal chance of being part of the sample
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Why use non-probability sampling?

A

There are no lists for some populations under study, for example:
- The homeless
- Certain occupations (e.g. farmers)
- Hidden or specific populations (e.g. farmers with mental health issues
- convenience/resource restriction

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Types of non-probability samples

A
  • convenience sample
    -snowball sample
  • Purposive sample
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What is a convenience sample

A

A sample of available participants
Example: students enrolled in a particular course or people passing a particular location

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What are advantages and disadvantages of convenience sampling

A

Advantages:
Easy
Inexpensive

Disadvantages:
No control over representativeness
Bias

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What is snowball sampling?

A
  • Involves collecting data with members of the population that can be located and then asking those members to provide information/contacts for other members of the population
  • Used mainly for hard-to-study populations
    for example: homeless young people, people with not commonly listed characteristics
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What is Quota sample?

A
  • Non-probability sampling equivalent of stratified random sample
  • Want to reflect relative proportion of population but you don’t/can’t sample randomly from each strata as you do in stratified random sampling
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What is purposive/judgment sampling?

A
  • Selecting a sample based on knowledge of the population, its elements and the purpose of the study
  • Clear purpose to sampling strategy. Select key informants, atypical cases, deviant cases or a diversity of cases

Example: If a study aimed to find problems experienced by new immigrants it may sample key people involved in agencies that help immigrants such as ethnic welfare groups, community immigration legal aid groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Why is purposive sampling often used?

A
  • To select cases that may be especially informative
  • Select cases in a difficult to reach population
  • Select cases for in depth investigation
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Which method of sampling do I use?

A
  • best method is normally a probability sampling one (as the aim of research is to generalise findings to the population)
  • Sometimes different sampling methods aren’t feasible given resources, time etc.

-

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

How do you determine sample size?

A
  • Largely determined by the analysis you plan to conduct with the data derived
  • Generally: the more complex the analysis the larger the sample
25
Q

When are larger sample sizes needed?

A
  • When the sample is heterogeneous (when the sample is composed of widely different people)
  • When you want to break down the sample into subcategories (e.g. look at gender separately)
  • If you want to obtain a more narrow or precise confidence interval
  • when you expect a small effect or weak relationship
  • for some statistical techniques
26
Q

What are the five simple rules for determining sample size?

A
  • If less than 100, use entire population
  • Larger sample sizes make it easier to detect an effect or relationship in the population
  • Compare to other research studies in area by doing a literature review
  • Use a power table for a rough estimate
  • Use a sample size calculator (e.g. G power)
27
Q

What is a metric? (levels of measurement)

A

When we want to measure something (e.g religion, self-esteem, tennis ability) we need to choose a metric with which we can measure it

The metric will determine the statistical analysis we will perform

28
Q

What are the levels of measurement?

A

Nominal
Ordinal
Ratio
Interval
(NORI)

29
Q

What is nominal?

A

Something which is purely categorical information (quality or kind of something)
Example: religion, gender

30
Q

What is ordinal?

A

A rank order
Ordinal variables do indicate an underlying quantity but they do not obey mathematical laws (you cannot meaningful subtract or divide)

31
Q

What is interval?

A

A true number in the sense that there are equal intervals implied but no true zero point - example: temperature in degrees

32
Q

Ratio

A

A true number. The distinguishing feature of a ratio scale variable is that it has a meaningful zero point, which participants could use to indicate the quantity is completely absent

33
Q

What is an issue with reliability and validity

A

The issue is you can’t assess these until after you have developed your questionnaire and use them . Therefore, a pilot test can be beneficial
Many people chose to use established measures instead of their own

34
Q

What are the two broad types of validity

A

Internal validity
External validity

35
Q

Why is validity an issue?

A
  • Many (if not most) variables in social research cannot be directly observed
    example: motivation, satisfaction, helplessness

Therefore, the challenge is to make a judgment call on weather we are measuring what we think we are measuring

36
Q

What are the types of validity

A

Face validity
content validity
criterion-related validity - (concurrent validity +predictive validity)
construct validity - (convergent + divergent)

37
Q

What is face validity

A
  • Asks the question: on the face of it, does my measure seem to relate to the construct?
  • Measures that lack face validity have the potential to alienate research participants (what are they really trying to measure)
  • a weak subjective method for assessing validity but a first step
38
Q

What is content validity

A
  • The extent to which the measure represents a balanced adequate sampling of relevant dimensions
  • Considers what should go into a measure and what should stay out - considers boundaries
  • How much does the measure cover the content of the definition?

Example: which of the following would be a more valid test of mathematical ability;
- a 20-question test containing addition problems
- a 20 question test containing addition, subtraction, multiplication problems

39
Q

What is criterion-related validity

A
  • involves checking the performance of your measure against some external criterion
40
Q

What are the two types of criterion-related validity:

A

Concurrent - Establish the validity of your measure by comparing it to a “gold standard” (i.e., Existing validated measure of the same construct)

Predictive - does the measure predict/relate to some criterion that you would expect it to predict

41
Q

What is predictive validity (in criterion-related validity)

A
  • Does the measure predict something that it is theoretically supposed to predict
  • Does the measure differentiate between people in a way that you would expect
  • What should a measure of the following constructs predict?
  • Iq -> perhaps some cognitive-based performance task
  • Workplace depression scale -> number of mental health sick days
42
Q

What is construct validity

A

Demonstrating that the measure relates to the theoretical construct of interest

43
Q

What are the two types of construct validity

A

convergent
divergent

44
Q

What is convergent validity (in construct validity)

A

Demonstrates that the measure relates to other similar measures

45
Q

What is divergent validity (in construct validity)

A

Demonstrates that the measure does not relate to unrelated constructs

46
Q

Summary of validity types

A

Face - in the judgment of others items appear to relate to construct

content - Captures the entire meaning (all elements of definition) of a construct

criterion - agrees with external source

concurrent - agrees with existing gold standard measure

Predictive- agrees with future behaviour

construct - how well multiple indicators relate to each other (consistent with theory)

convergent: similar measures (or measures of theoretically related constructs) are related

divergent - different measures are unrelated

47
Q

What is reliability?

A
  • The consistency or repeatability of your measurement
48
Q

What are the types of reliability?

A
  1. Stability of the measure (test-retest)
  2. Internal consistency of the measure (split-half,
    Cronbach’s alpha)
  3. Agreement or consistency across raters (inter-rater)
49
Q

What is test-retest reliability?

A
  • Addresses the stability of your measure
  • you administer the measure at one point in time and then you give the same measure to the same participant at a later point in time
  • You correlate the scores on the two measures
50
Q

What is the problem with test-retest?

A
  • memory effect
  • practice effect (performance improves because person has had practice taking test before)
  • Another thing to consider is time between intervals. If too short there is a greater risk of memory effect. If too long there is a risk of other variables (e.g. additional learning) influencing results
51
Q

What is split half reliability

A
  • administer a battery of questions
  • split the measure into two halves
  • correlate the score on the two halves of the measure

higher correlation means greater reliability
strengths: eliminates memory and practice effects
limitations: are the two halves equivalent

52
Q

What is inter-item reliability

A
  • Assesses the internal consistency of the measure
    ie. tells you how well items or questions in your measure appear to reflect the same underlying construct.
  • You will get good internal consistency if individuals respond in approximately the same way on your survey

Cronbach’s alpha can range from 0 (when the items are not
correlated with one another) to 1.00 (when all items are
perfectly correlated to each other). The closer the alpha is
to 1.00, the better the reliability of the measure

53
Q

Inter-rater or inter-observer reliability

A
  • Checking the match between two or more raters or judges in your study
54
Q

Calculations of inter-rater reliability

A
  • Nominal or Ordinal scale (the percentage of times different raters agree)
  • Interval or ratio scale - correlation coefficient
55
Q

What kind of reliabilities co-efficients should I be
aiming for?

A
  • Test-retest coefficients > .70
  • Internal consistency >.70 (but ideally much higher)
  • Rating consistency >.90
56
Q

Reliability and measurement error

A

*measurement error serves to weaken our statistical tests

  • All other things being equal, more error in measurement means lower power
  • Choosing a measure which is highly reliable decreases measurement error and increases the power of your design
57
Q

Can a measure be reliable but not valid?

A

Yes! You could have a consistent measure that does not
actually measure the construct

58
Q

Can a measure be valid but not reliable?

A

Yes.
Example of valid tool but is unreliable – something that is
difficult to implement (e.g., Skin fold tests – require
technical skill) – may be unreliable across multiple
administrators.

59
Q

Summary of validity types

A

Test-Retest - Same question given on different occasions and data correlated

Split half - Split questions in half and correlate data from two halves

Inter-item reliability - Overall correlation between items in scale

Inter-rater - Checking for agreements between multiple raters or judges

Pyb210 - Statistics (11 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions