Research methods - Review

Question 1

What is chronbach’s alpha and what does it assess? what score on cronbach’s alpha is acceptable?

Answer 1

the average correlation among all possible pairs of items. (easily calculated [by a computer] and reported) Cronbach’s alpha can range from 0-1. SPSS will calculate it and can also indicate the change in Cronbach’s if any particular item were omitter. Good (i.e., acceptable, though not outstanting reliability = .80*. the higher the better) internal reliability

Question 2

What do 95% confidence interavals show/mean/represent?

Answer 2

they define the range of values within which the true mean is likely to lie on 95% of occasions. 95% CI indicate error of measurement (in estimation of population mean) whether means are likely to differ significantly

Question 3

if the confidence interval bars for two means overlap by more than 1/4 is the difference between means likely to be significant or non-significant?

Answer 3

Non-significant

Question 4

What is significance?

Answer 4

The significance level or ‘p’ is the probability that an observed effect arose by chance if there really is no effect.

Question 5

What are the three types of descriptive statistics and when are they used?

Answer 5

• Mean (M), median & mode - tell us typical score on a continuious measure for comparing means between conditions.
• Percentages or proportions of cases that fall into particular categories (or frequencies) - compare between catefories with different no. of cases.
• Correlation coefficients (r) and regression coefficients (B and beta) - describe the nature of a linear relationship between continuous variables. direction and strength. Bs tell us about the effect in the original scales of the measurement. Betas can indicate the relative strength of different predictors.

Question 6

parametric tests assume that…….?

Answer 6

• scores on continuous variables (that are used in the analysis) are normally distibuted
• the variability (or variance) in scores in similar for different conditions (there are tests available to assess this, including in SPSS if these assumptions are violated, then the outcome of parametric test can be misleading. If the data do deviate markedly from these assumptions, we could use a nonparametric test.

Question 7

What are the broad classes/types of parametric tests?

Answer 7

• tests that assess the signifiance of the difference between means (t-test and ANOVAs)
• Tests that assess the significance of the linear relationship between continuous variables (correlations and regressions)

Question 8

What are the assumptions of an ANOVA/t-test?

Answer 8

• Scores on the DV are (roughly) normally distributed) i.e., bell curve)
• Variance or variability is (roughly) similar for the different conditions(the assumption of ‘homogeneity of variance’)

Question 9

In order to meet the homogeneity of variance assumption (i.e. in ANOVA) the levene’s test should be _____________?

Answer 9

Non-significant

Question 10

In what situations does heterogeneity of variance(different levels of variance between conditions) pose a greater threat to the validity of the statistical outcome?

Answer 10

• the n per condition is small the Ns per condition are markedly different
• the direction of skewness varies between conditions
• Heterogeneity of variance and non-normality coexist.

Question 11

What is the appropriate parametric test to assess the difference between means where the IV is varied between subjects and there is one IV with 2 levels?

Answer 11

Independent groups t-test (or one-way ANOVA)

Question 12

What is the appropriate parametric test to assess the difference between means where the IV is varied between subjects and there is one IV with 3 levels?

Answer 12

Oneway ANOVA

Question 13

What is the appropriate parametric test to assess the difference between means where the IV is varied within subjects and there is one IV with 2 levels?

Answer 13

Paired samples t-test

Question 14

What is the appropriate parametric test to assess the difference between means where the IV is varied within subjects and there is one IV with 3+ levels?

Answer 14

Oneway repeated measures ANOVA

Question 15

What is the appropriate parametric test to assess the difference between means where the IV is varied between subjects and there are 2 IVs with 2+ levels on each?

Answer 15

Factorial ANOVA (2-way, 3-way etc)

Question 16

What is the appropriate parametric test to assess the difference between means where the IV is varied within subjects and there are 2 IVs with 2+ levels on each?

Answer 16

Repeated measures ANOVA

Question 17

What is the appropriate parametric test to assess the difference between means where the IV is varied both between and within subjects (i.e., mixed design) and there are 2 IVs with 2+ levels on each?

Answer 17

mixed ANOVA (split plot ANOVA)

Question 18

What is a moderator?

Answer 18

The moderator is the variable (another IV) that affects the primary IV-DV relationship in which we’re interested. In other words, the moderatory affects the relationshop between the key IV and the DV. We say that two IVs interact. We could just as correctly say that the IV and the moderator interact. (Moderation is closely related to interaction, however it’s a more theorestical idea in that it reflects our focus in posing and answering a research question)

Question 19

What is an Interaction?

Answer 19

The term interaction indicates that the effect of one IV varies, depending on another IV. It applies when we expect that the size and or direction of the effect of one IV will differ depending on the other IV. A difference in size means that an IV has a larger effect at one level of the 2nd IV than at the other level. A difference in direction means that an IV has opposite effects at the 2 levels of a 2nd IV. An interaction can be a conceptial proposition, but we also test the statistical significance of interactions.

Question 20

The following hypotheses is suggestive one what kind of relationship between IVs?: It was predicted that information type and coping style would interact in their effects on pre-operative anxity. Specifically, it was expected that, for people with a problem-focused style, detailed infroamtion would result in lower anxiety than would routine information. In contrast for those who use avoidant coping, it was expected that routine information would result in lower anxiety than would detailed information.

Answer 20

an INTERACTION

Question 21

What two types of effects do factorial ANOVAS (2+ IVs) test?

Answer 21

Main effects of each IV Interaction between the IVs. e.g., whether the difference in the mean anxiety between detailed and routine information itself differs between problem-focused people and avoidant people. That is, does the difference between detailed and routine vary between the coping styles? or, in other words, is detailed minus routine different for problem focused people than it is for avoidant people?

Question 22

What is a main effect?

Answer 22

A test of one IV at a time A main effect refers to the overall effect of one IV, completely ignoring the other IV. For example, the test of the main effect of coping style indicates the significance of the overall difference between problem-focused and avoidant participants, collapsed or averaged across levels of information type. That is, all of the problem-focused participants are compared with all of the avoidant participants (regardless of the type of information that thye received). Ps are distinguished only according to their coping style. When an interaction is significant, main effects are sometimes of limited interest because they don’t tell the whole story. We report them, but the interaction would be a central result.

Question 23

What is moderation?

Answer 23

Moderation is closely related to interaction (moderation is only demonstrated if an interaction exists). However, it’s a more theorestical idea in that it reflects our focus in posing and answering a research question. An interaction indicates that the relationship between an IV and the DV varied depending on another variable. That other variable is often referred to as the moderator. So the MODERATOR is the variable (another IV) that affects the primy IV-DV relationship in which we’re interested, In other words, the moderator affects the relationship between the key IV and the DV. We say that two IVs interact. We could just as correctly say that the IV and the moderator interact.

Question 24

There are two advertisments for mobile phones, one is ‘transformers’ and one is ‘connecting people’, the phone company wants to know which one is the better ad. You might think that the preferred advertisement will vary depending on the gender of the teenager. If you do think this, then you are predicting an interaction between type of advertisement and sex of teenager or in other words that…. Sex of teenager _______fill blank the effect of type of advertisement.

Answer 24

Moderates

Question 25

Which of the following is not a potential threat to the internal validity of a quasi-experiement? A. Instrumentation B. Generalisability C. History D. Selection E. Maturation

Answer 25

B. Generalisability (because this concerns external validity!)

Question 26

2. A threat to internal validity of a static group comparison (non-equivalent control group) design is: A. Selection B. Instrumentation C. Testing D. Regression toward the mean E. Response bias     Also, describe a static group comparison design.

Answer 26

A. selection   Group 1    X    O    Group 2           O    (control)   why not the others?: B. Instrumentation - no reason to think this C. Testing - no repeated measures D. Regression toward the mean - repeated measures only E. response bias - no idea about this and why would it differ

Question 27

Research has ________ validity when the measures provide a very good index of the phenomena that are being studied A. Internal B. External C. Construct  D. Discriminant E. None of the above

Answer 27

C. Construct

Question 28

A correlation of .3 between anger and aggressive behaviour: A. Indicates that anger causes agression B. Indicates that anger explains 9% of the variance in aggressive behaviour C. Indicates that reducing anger will reduce aggressive behaviour D. Shows that anger and aggression are strongly related. Although we can't be sure of the causal direction E. Shows that another variable is responsible for the ange-aggression relationship

Answer 28

B. Indicates that anger explains 9% of the variance in aggressive behaviour

Question 29

For samples of a given size, external validity is promoted by A. Non-probability samples B. volunteer samples C. Random samples D. Quota samples E. Samples of convienience

Answer 29

C. Random samples

Question 30

Consider these questions Q1 “How close is your relationship with your mother?”  - Very close, quite close, not close, quite distant, very distant  Q2 “What was the $ value of your last telephone bill?” _______ Q3 “Australia should remain a monarchy because this is the only way to ensure that the political power of the government is held in check to safeguard the rights of citizens” - strongly disagree, disagree, unsure, agree, strongly agree How would we evaluate the quality of the questions? A. Q1 contains an unwarranted assumption; Q2 is ambiguous; Q3 would be vulnerable to social desirability effects B. Q1 is satisfactory; Q2 is too sensitive - the amount of money spent on phone bills is none of our business; Q3 is satisfactory C. Q1 is too sensitive - we should not ask people questions about their mothers; Q2 is good because it asks for specific information; Q3 is leading D. Q1 contains an unwarranted assumption; Q2 requires information that the participant might not remember; Q3 is double-barrelled E. all of the questions are satifactory

Answer 30

D. Q1 contains an unwarranted assumption; Q2 requires information that the participant might not remember; Q3 is double-barrelled.

Question 31

First consider this 2x2 between groups desgin study: Researchers investigated the effects of ‘author’s sex’ and ‘errors’ on judgements that people make about the quality of a journalist’s writing. In the study, participants were randomly allocated to one of four conditions: (i) female author and grammatical errors, (ii) female author and no grammatical errors, (iii) male author and errors, (iv) male author and no errors. Author and errors were manipulated using bogus journalist names and 4 similar articles. The participants read the newspaper article and were asked: “What is the quality of the newspaper article?” (rated on a standard rating scale). Q1. What is the purpose of question “What is the quality of the newspaper article?” A. It provides a measure of the independent variable B. It provides a measure of the dependent variable C. It provides an assessment of extraneous variables D. It is a manipulation check E. Both (C) and (D)   Q2. The above design is: A. a true experimental, posttest only two-group design with one between-subjects factor and one within-subjects factor B. a true experimental, posttest only two-group design with two between-subjects factors C. a quasi-experimental, static group comparison with two between-subjects factors D. a true experimental, four-group design with one-between subjects factor and one within-subjects factor E. a 2 x 2 between-subjects factorial design

Answer 31

Q1: B. It provides a measure of the dependent variable Q2. E. a 2 x 2 between-subjects factorial design

Question 32

The correlation coefficient for the scatter plot is most likely to be (and why) A. 1.00  B. .64 C. -.80 D. -1.00 E. Impossible to say without knowing the degrees of freedom 

Answer 32

C. - 80   Graph goes right to left = negative

Question 33

In a study of the number of crimes per year and the number of churches in towns & cities... CRIMES and CHURCHES were found to be highly correlated, r = .69. However, when the population size (POP) of the towns and cities was taken into account, the partial correlation was .11 (nonsig). This gives us good reason to conclude that: A. There are likely to be other unstudied factors which predict CRIMES more strongly than POP or CHURCHES (??) B. CRIMES is highly related to CHURCHES but not to POP (??) C. There is not a strong linear relationship between CRIMES and CHURCHES (the correlation is strong) D. The relationship b/t CRIMES & CHURCHES is a spurious one E. At least three of these (A, B, C, D) are true

Answer 33

D. the relationship between Crimes and Churches is a spurious on.

Question 34

Which of the following is true of the figures above? A. Figure 1 shows heteroscedasticity; Figure 2 shows a strong negative correlation, Figure 3 shows an interaction B. Figure 1 shows homoscedasticity; Figure 2 shows a strong negative correlation, Figure 3 shows main effects but no interaction C. Figure 1 shows heteroscedasticity; Figure 2 shows a weak negative correlation, Figure 3 shows an interaction D. Figure 1 shows heteroscedasticity; Figure 2 shows a moderate positive correlation, Figure 3 shows an interaction E. Figure 1 shows heteroscedasticity; Figure 2 shows a moderate positive correlation, Figure 3 shows main effects but no interaction

Answer 34

A. Figure 1 shows heteroscedasticity; Figure 2 shows a strong negative correlation, Figure 3 shows an interaction.

Question 35

What DOES and DOESN'T the significance value (p) tell us?

Answer 35

DOES: tell us if a relationship exists Doesn't: tell us effect size or give any conceptual understanding, or how compelling the evidence is

Question 36

What three things can tell us something about the meaning of our results?

Answer 36

Effect size Clinical significance Error bars

Question 37

p (sig. level) depends on the number of _________?

Answer 37

Participants (N)

Question 38

Variance explained and the size of difference between means are examples of?

Answer 38

Formal effect size indices.

Question 39

What is variance explained?

Answer 39

Proportion of total variance in the DV that is explained by the effect (IV)  

Question 40

What are the indices for total variance explained for A.ANOVA B. Regression and C. correlation

Answer 40

Eta2 (n2) R2 r2 (regression coefficient squared)

Question 41

eta2 and R2 are both indices of proportion of variance explain and are the SAME THING - but which tests are they reported for by convention?

Answer 41

Eta2 - for ANOVA R2 - for regression

Question 42

SPSS does not provide eta 2, so it is hand calculated....how?

Answer 42

SS (i.e., sum of sqaures) effect / SS corrected total (NOT TOTAL!!)

Question 43

What is the difference between R2 and R2change ?

Answer 43

Both are indices of proportion explained....but R2 is is total variance explained and R2change is the change for a particular step (i.e. additional variance explained) in hierachical regression.

Question 44

What is the difference between partial eta2 and eta2 (factorial ANOVA)

Answer 44

Partial eta2 ignores any other factors or effects in an analysis and only considers SSerror (unexplained variability) and any extra variability form the effect of IV (SSeffect). thus it can't be used to compare effect sizes (it is not standardised). Only MATTERS when there is more than one factor. (if only one factor eta2 = partial eta2)   Eta2 is standardised and can be used to compare and judge effect sizes.

Question 45

cohens d is a ___________ measure of the difference between ______ and is expressed in ________ units

Answer 45

effect size; means; standard deviation

Question 46

how it cohen's d calculated?

Answer 46

(m1-M2)/ ___________________ (SD1 +SD2)/2 (i.e., average SD for the means)

Question 47

What are the benchmarks for eta2 effect size measures? (small/medium/large)

Answer 47

small - .010 -.04  (1%) medium - .06 - .130 (6%) large - .14 + (14%)

Question 48

What are the benchmarks for R2 effect size measures? (small/medium/large)

Answer 48

Small- .02 (2%) medium - .13 (13%) large - .26 (26%)

Question 49

What are the benchmarks for r effect size measures? (small/medium/large)

Answer 49

small - .1 (1%) medium - .3 (9%) large - .5 (25%)

Question 50

What are the benchmarks for d (cohens) effect size measures? (small/medium/large)

Answer 50

small - .20 medium - .50 large - .80

Question 51

What are the problems with relying on a table/conventional benchmarks for effect size measures?

Answer 51

* Never intened them to be indiscriminately applied * Mainly intended for new areas of research and deciding on the necessary sample size (power analysis) when study being planned. * were based on 'typically' observed effect sizes for various analyses across numerous behavioural size areas, not on what is important/useful * they are non-equivalent for different indices (e.g., eta 2 and R2 - even though they are effectively the SAME!)

Question 52

small variance explained by interactions can still be _______

Answer 52

important.

Question 53

What are the three types of error bars?

Answer 53

* standard deviations (indicated variability in scores- least common) * standard erros (allows statistical inference) * 95% confidenc eintervals (allows statistical inference)

Question 54

Briefly outline the ethics of research with humans.

Answer 54

Participants' rights outweigh other considerations.  Participation must be voluntary Consent must be informed Participants may withdraw at any time Participants response and performance must remain confidential - they also must remain anonymous Harm to participants should be minimised and provisions to adress any stress etc should be done Must have the approval of ethics committee Participants should be debriefed

Question 55

What are closed-ended questions?

Answer 55

questions that provide respondents ith a fixed set of alternatives from which to choose

Question 56

What are rating scales (for questionaires)?

Answer 56

Rating scales include Likert scales. Usually 4-7 response alternatives e.g., 5-point rating scale from 1 "strongly disagree" to 5" strongly agree. Strictly speaking, Likert scales assess the extent of the agreement with a statement and so, have anchors such as "Agree" and "Disagree". However, there are many variations on this theme, such as "approve- disapprove" or "Satisfied- dissatisfied", which are simply referred to as 'rating scales' (or Likert-type scales)

Question 57

What is the difference between unipolar, bipolar and sematic differential scales?

Answer 57

Unipolar - a rating (e.g., 1-6) towards either ends of one construct e.g. satisfaction (1 not at all satisfied -----6 very satisfied Bipolar - a rating (e.g.., 1-6) towards either ends of two constructs e.g., 1 very unsatisfied to 6 very satisfied. Semantic Differential Scales - anchors are polar opposite adjectives, e.g., healthy-sickly, generous-selfish etc

Question 58

What are some qualities that make questionaires "good"?

Answer 58

Ensure that all possible response alternatives are available Use clear verbal anchors for rating scales Use more, rather than fewer, question to assess a construct (such as self-esteem). (Reliability tends to be greater with more questions because, if there's a poor question it has less effect._ NOTE: response alternatives must match the question. Use existing, validated, questionnaires where possible If designing your own questions, pilot or pre-test them (begin asking friends to read the questions: what appears to be a model of clarity to the author can seem exceedingly muddy to the naive respondent) Determining and reporting reliabilities for questionnaire data is a standard procedure (where relevant or appropriate, e.g., it usually makes no sense to determine reliabilities for basic demographic information)

Question 59

What are some potential threats/problems with measurement validity and/or reliability?

Answer 59

Response biases - social desirability, response sets (acquiescent response set, deviation response set) Other problems Unclear question or instructions (instructions are important) Appropriate response not available Imprecision of many ordinal scales (differences in interpretation) Question is double-barrelled (or, much worse, quadruple-barrelled!) Respondent doesn't know the answer (e.g., can't remember) Suggestibility: the respondent did not really hold a view on an issue until the questionnaire suggested it. A question might be based on an inappropriate and unwarranted assumptions about respondents (e.g., not applicable) Leading questions Very general questions Sensitivity of an issue.

Question 60

What is the problem with this questionaire question: I am not againt Australia remaining a monarchy: strongly agree      agree    don't know    disagree

Answer 60

Double negative; unbalanced scale

Question 61

What is the problem with this questionaire question: Experts agree that compulsory vehicle inspection would save lives. To what extent do you agree that compulsory vehicle inspection should be instituted in this state? (leading) 1 strongly agree   2 agree    3 neither agree nor disagree   4 disagree  5 strongly disagree

Answer 61

Leading question

Question 62

What is the problem with this questionaire question?: Finacial assistance for the arts should be reduced so that the funds could be used to attract business to the state 1 strongly agree   2 agree    3 neither agree nor disagree   4 disagree  5 strongly disagree

Answer 62

Double barrelled

Question 63

What is the problem with this questionaire question?: Have you ever left a small child unattended in a car for 30minutes or more?  Yes   No

Answer 63

Sensitive question

Question 64

What is the problem with this questionaire question?: Are you Australian? Yes   No

Answer 64

What does 'Australian" mean?

Question 65

What is the problem with this questionaire question?: Is your occupation....___blue collar or ___white collar 

Answer 65

No other categories? Vague categories. Respondent might not work.

Question 66

What is the problem with this questionaire question?: How many times have you shopped in Rundle Mall in the past six months? _____times

Answer 66

Will participants remember?

Question 67

What is the problem with this questionaire question?: Are you currently employed? ___Employed  ____Unemployed

Answer 67

Response options don't match the question & is this an exhaustive list? student? retired? parent?

Question 68

What is the problem with these questionaire question?: What do you usually have for breakfast? Do you eat a healthy diet? ______

Answer 68

'usually' is too vague, what does 'healthy' mean?

Question 69

What is Standard Deviation (SD)?

Answer 69

The SD can be thought of as the average amount by which any score (in a set of scores) differs from the mean. (It's actually a bit bigger than the strict average). It tells us how typical or atypical the mean is of scores within a group, and how much, on average, scores deviate from the mean (above and below). It uses the same unit of measurement as that which was used to measure the variable. SD is the square root of the variance  

Question 70

What is effect size?

Answer 70

Effect size simply refers to the strength (often called a magnitude) of an effect (or of a relationship or difference), i.e., how big the effect is. It usually tells us something about the importance of a relationship (or of a difference between groups, which is also a relationship)

Question 71

Can the difference between means be intepreted as an effect size?

Answer 71

Yes, if we are comparing means for different conditions (or times), we can assess effect size just by looking at the degree of difference between means. (can also use cohen's d) For example, if we reduce the number of times that autistic children engage in head-banging from a baseline mean of 26 times per day to a post-treatment mean of 1 time per day, this is much more impressive (larger effect of treatment) and useful outcome than if we'd reduced it from 26-20 times a day.

Question 72

Does the size of a correlation coefficient indicate the strength of the linear relationship between two variables?

Answer 72

Yes, the size of the correlation indicates the strength of the relationship but its significance level does not. (because very small  correlations can attain significance with sufficient participants!)

Question 73

What is reliability?

Answer 73

Consistency or stability of a measure. Logically, if a measure is valid (a good/accurate measure), it should be reliable.

Question 74

What is internal reliability?

Answer 74

The degree of correspondence/agreement between individual items that make up a measure of a single characteristic. Logically, if a number of items are meant to measuring the same construct, there should be a high level of agreement between them. Cronbach's Alpha: the average correlation questionnaires (easily calculated [by a computer] and reported) Cronbach's alpha can range from 0-1. SPSS will calculate it and can also indicate the change in Cronbach's if any particular item were omitter. Good (i.e., acceptable, though not outstanting reliability = .80*. the higher the better)

Question 75

What is item-total correlation (reliability)?

Answer 75

The correlations between each item and the total score (of a measure).

Question 76

What is split-half reliability?

Answer 76

The correlation between total scores for the two halves of a test of measure.

Question 77

What is test-retest reliability?

Answer 77

the correlation between scores for 2 administrations of the measure.

Question 78

What is inter-score or inter-rate reliability?

Answer 78

The correspondence or agreement between the scores of two independent observers or judges. Used when judgement is used by the score or assessor, or where assessment requires skill or training.

Question 79

What is a Factorial design?

Answer 79

Where 2 or mor ecategorical IVs are manipulated (or measured if it is a quasi-experiment) AND all combinations of the IVs are tested.   allows us to see both the separate and combined effects of the IVs.

Question 80

what is "nominal' level of measurement?

Answer 80

Categorical information (e.g.., employer status, sex)

Question 81

what is "Ordinal' level of measurement?

Answer 81

when the 'quantification' of th econstruct is imprecise, but we do assume that the scores indicate something about the relative amounts of a particular characteristis, that is, that we could rank scores in a meaningful order (e.g., 0-5 stars on a movie)

Question 82

What is 'interval' level of measurement?

Answer 82

A more precise measurment, where the difference (or distance or interval) between any two adjacent scores in the same and the unit of measurment does have a universal maning and indicates the same amount of a property across different participants or situation.   However the scale does not have an absolute zero.   e.g., IQ score, temperatur

Question 83

what is "ratio' level of measurement?

Answer 83

Precise measurement/quantification whereby 0 means NONE/abscense of the characteristic   e.g., heartrate, number of children, number of words recalled

Question 84

What are nonparametric tests used for?

Answer 84

Continuous DVS that use an ORDINAL DEPENDANT VARIABLES (i.e., they can be put in rank order but there is not equal distance between each value and no true 0)   +   data that does not meet the assumptions for patametric tests (i.e., skewed data)

Question 85

chi-squares are used for what type of dependant variable?

Answer 85

Categorical DV

Question 86

The Phi is an effect size measure for what test and how is it intepreted?

Answer 86

chi-square - intepretted like a correlation.

Question 87

nonparametric tests for continuious DV use _____ rather than ______

Answer 87

ranks; raw scores

Question 88

When the distribution of scores on a continuous DV is skewed what  descriptive state is used?

Answer 88

Median (middle score)