week 2 Flashcards
measuring variables, sampling, validity and reliability
what are generalisable results
results that are deemed to reflect the true state of affairs in the population of interest
how do you claim generalisability?
your sample needs to be as representative of the population as you can make it
types of sampling procedures
non-probability sampling
probability sampling
types of non-probability sampling
convenience
snowball
purposive
types of probability sampling
simple random sample
systematic random sample
stratified random sampling
multi-stage cluster sampling
how do you ensure your sample is representative of the population
a sample will be representative if all members of the population have an equal chance of being selected in the sample
what does probability sampling allow
the researchers to calculate the relationship between the sample and the population
what is a simple random sample
each member has an equal and independent chance of being selected
how do you do simple random sample
define the population
list all members
assign numbers
eg.
- use a table of random numbers to select
- use a lottery method
- use a computer program to randomly select
what is systematic random sampling
selects a random starting point from the population, then a sample is taken from regular fixed intervals of the population depending on its size
eg. to select a sample of 1000 people from a list of 10,000, randomly select the first person and then select every 10th person from the list
what is stratified sampling
researchers divide subjects into subgroups called strata based on characteristics that they share
eg. age, ethnicity, location
why do we use stratified sampling
can reduce sampling error by ensuring ratios reflect subpopulations
to ensure that small subpopulations are included in the sample
what is multi-stage cluster sampling
you draw a sample from a population using smaller and smaller groups at each stage
what is non-probability sampling
not every member of the population has an equal chance of being part of the sample
difference between stratified sampling and multi-stage cluster sampling
it is not the same as stratified sampling as each cluster doesnt need to be sampled
why would we use non-probability sampling
because not every member of the population has an equal chance of being part of samples
eg. homeless people
what is convenience sampling
choosing people who are easy for the researcher to reach and get in touch with
eg. students enrolled in a particular course
pros and cons to convenience sampling
pros
- easy
- cheap
cons
- no control over representativeness
- bias
what is snowball sampling
Involves collecting
data with members of
the population that
can be located and
then asks those
members to provide
information/contacts
for other members of
the population
why is it used
to study hard to reach populations
eg. homeless youth
QUT students who use the library at night
what is quota sampling
a non-probability sampling method in which researchers create a convenience sample involving individuals that represent a population. Researchers choose these individuals according to specific traits or qualities.
what is purposive/judgment sampling
Selecting a sample based on knowledge of the population, its elements, and the purpose of the study
why do we use purposive/judgement sampling
it is often used to:
- select cases that might be especially informative
- select cases in a difficult to reach population
select cases for in-depth investigation
for quantitative research which type of sampling should be used
probability sampling
when are larger sample sizes needed
when the sample is heterogeneous
when you want to breakdown the samples into subcategories
if you want to obtain a narrow or more precise confidence interval
when you expect a small effect or weak relationship
for some statistical techniques
5 rules for determining sample size
- if less than 100 use entire population
- larger sample sizes make it easier to detect an effect or relationship in the population
- compare to other research studies in area by doing a lit review
- use a power table for a rough estimate
- use a sample size calculator eg. g power
what is a metric
a measure that is quantifiable
what will the metric determine
the statistical analyses we can perform
4 levels of measurement
nominal
interval
ordinal
ratio
what is a nominal measurement
Not a quantity, but rather
a discrete quality that
something can have
something which is purely categorical information
eg. religion
what is an interval measurement
a true number in the sense that there are equal intervals implied, but no true zero point
eg. temperature in degrees
what is ordinal measurement
a rank order
what is a ratio measurement
a true number. the distinguishing feature of a ratio scale variable is that it has a meaningful zero point, that participants could use to indicate the
quantity is completely absent
what is validity
how well the results among the study participants represent true findings among similar individuals outside the study
types of validity
face validity
content validity
criterion validity
- concurrent validity
- predictive validity
construct validity
- convergent validity
- divergent validity
what is face validity
the degree to which the study appears effective in terms of its stated aim eg. measures what it said it would
what do measures that lack face validity have the potential to do
alienate research participants
what is content validity
Consider what should go into a measure, and what should stay out
- define the boundaries
whats the difference between face and content validity
Face validity is an informal review of a questionnaire by non-experts, who assess its clarity, comprehensibility, and appropriateness for the target group
content validity involves a formal assessment to determine the appropriateness of content
what is criterion-related validity
it involves checking the performance of your measure against some external criterion
types of criterion-related validity
concurrent: does it relate to a known criterion
eg. an alternative gold standard measure of the same construct
predictive: does the measure predict/relate to some criterion that you would expect it to predict
what is concurrent validity
does our measure agree with pre-existing gold standard measures
what is predictive criterion validity
does our measure agree with theoretically future behaviour?
what is construct validity
How well the measures align with the theory
types of construct validity
convergent
divergent
what is convergent construct validity
demonstrating that the measure relates to measures of similar and related theory
what is divergent construct validity
demonstrating that the measures does not relate to unrelated constructs
what is reliability
the consistency or repeatability of your measurement
types of reliability
stability of the measure (test-retest)
internal consistency of the measure (split-half, cronbachs alpha)
agreement or consistency across raters (inter-rater)
main problems with test-retest
memory effect
practice effect (practice improves because of practice in test taking)
what is test-retest reliability
you administer the measure at one point in time. you then give the same measure to the same participants at a later point in time. you correlate the scores on the two measures
what is split half reliability
administer questionnaires and split the measures into 2 halves. correlate the scores on the 2 halves of the measure. higher correlation means greater reliability
strengths and limitations to split-half reliability
strength
- eliminates memory and practice effects
limitations
- are the 2 halves equivalent
what is inter-item reliability
assesses the internal consistency of your measure
eg. tells you how well the questions in your measure appear to reflect the same underlying construct
inter-rater/ inter-observer reliability
checking the match between 2 or more raters/judges
eg. research investigating the relationship between communication and family functioning
calculation of inter-rater reliability for nominal or ordinal scale
the percentage of times difference raters agree
calculation of inter-rater reliability for interval or ratio scale
correlation coefficient
reliabilities coefficient scores for testretest, internal consistency and rating consistency
test-retest coefficient >.70
internal consistency >.70 but aim for more
rating consistency >.90
Can a measure be reliable but not valid?
yes
you could have a consistent measure that does not actually measure the construct
Can a measure be valid but not reliable?
yes
eg. something
that is difficult to implement (e.g., Skin fold tests –require technical skill) – may be unreliable across multiple administrators
