Experimental Design Flashcards
what are factorial designs?
experimental designs with more than one independent variable
what does the term “factor” refer to?
each manipulated independent variable
what are two-factorial designs?
experiments with two independent variables
what are experiments with two independent variables called?
two-factorial designs
what are factorial designs describes by?
described by the number of factors (independent variables) and the number of levels in each factor
A two-factorial design with two factors having two levels each is described as a …
2 x 2
what do the number of conditions correspond to?
the number of cells in the design
what are the different factorial designs?
- within-subject designs
- between-subject designs
- mixed designs
what is an interaction in factorial design?
In a factorial design, an interaction is said to occur when the effect of one independent variable
depends on the level of another independent variable.
what is a cross-over interaction?
In a cross-over interaction the effect of one variable reverses as a function of the other variable.
Furthermore, there is no main effects
what is a survey?
A survey is a structured set of questions or statements given to a group of people to measure their
attitudes, beliefs, values, or tendencies to act.
examples of questionaires
- paper
- online
- mailed
how can interviews be held?
- Live
- Telephone
what can be used for data acquisition for a survey?
- questionnaire
- interview
name formats of questions
- open-ended question
- closed question
what is an open-ended question?
participants provide narrative information
what is a closed question?
Participants should respond to pre-formulated answers.
what are the advantages of open-ended questions?
- eliciting a wide range of responses, including some not even conceived of by the researchers
- increase the sense of involvement of the respondents
what are the disadvantages of open-ended questions?
- open-ended questions are difficult to score
- take a lot of time to analyse
when are open-ended questions useful?
- asking an open-ended question at the end of closed questions
- using open-ended questions in a pilot study to get an idea of potential answers
types of closed questions?
- simple answer
- likert scale
what is an advantage of closed questions?
- Can be summarized in a quantitative way (e.g. means, standard deviation)
what should be considered when using closed questions?
- number of answer options (three, five, seven)
- some items should be formulated favourably and other unfavourably
- sequence of items (sensitive questions towards the end)
when should demographic information be added and why?
Should be added at the end (not to bore participants)
what are example of demographic information?
age, gender, socioeconomic status, marital status, education
what does the type of demographic information depend on?
the research question
what should you avoid when wording survey questions? (3)
- ambiguity
- double-barreled questions
-leading questions
what are double-barreled questions?
Question that touches upon more than one issue, yet allows only for one answer.
what are leading questions?
A question that is structured in a way that makes it likely to produce an answer desired by the
asker.
3 types of leading questions
- Leading questions that are based on assumptions
- Leading questions based on interconnected statements
- Leading questions based on coerciveness
what is the researcher’s goal with surveys?
try to draw conclusions about the population based on the sample
which sampling procedures should be used?
One should use probabilistic sampling procedures
what does the sample need to be/should not be?
It is crucial that the sample is representative and not biased.
what is psychometrics?
Psychometrics is a scientific discipline concerned with the question of how psychological constructs
(e.g., intelligence, depression) can be optimally related to observables (e.g., outcomes of psychological tests)
general steps of scale construction (8)
- Determine what you want to measure (e.g. paranormal beliefs)
- Generate an item pool
- Content adequacy assessment
- Questionnaire administration
- Factor analysis
- Internal consistency assessment
- Construct validation
- Replication
determine what you want to measure (4)
- definition of the construct
- objective of the scale
- population
- content of the scale
what are the two general approaches to item generation?
- inductive approach
- deductive approach
what is the inductive approach?
- generate a pool of items first
- use statistical methods to generate scales (e.g. factor analysis)
when is the inductive approach used?
used when investigating an unfamiliar topic
what is the deductive approach?
uses a theoretical definition of a construct which is then used as a guide for the creation of
items
when is the deductive approach used?
when clear theoretical assumptions about the construct exists
what should one be aware of when generating the items?
when generating the items one should be aware of the response format (e.g. Likert scale)
what is the goal of the item generation process?
The goal of the item generation process is to generate much more items than are needed for the final scale
because one will select the best items based on their psychometric properties.
what should you think about when generating items?
- redundancy
- writing process
- positive and negative wording of items
what are bad items (4)?
- Exceptionally lengthy
- Unnecessarily wordy
- Multiple negatives
- Double-barreled items
what are good items?(2)
- Unambiguous
- Targets the appropriate reading level for the intended sample
what is content adequacy assessment?
assesses whether items are really adequate for a specific construct
common approach of content adequacy assessment
- categorize or sort items based on their similarity to construct definitions
- respondents are presented with construct definitions without titles and are asked to match
items with a corresponding definition - can be done by experts or by non-experts
- maximizes content validity
how big should the sample size at least be?
Sample size should be at least 100 to 150 participants depending on the statistical approach
what does the sample size also depend on?
The sample size also depends on the number of items (more items -> larger sample).
what is a factor analysis?
Is a statistical method used to describe variability among observed, correlated variables in terms of
a potentially lower number of unobserved variables called factors.
what is the general goal of the factor analysis?
to determine the structure of a specific construct and
investigate which items are most strongly related to a specific scale or subscale
what is an exploratory factor analysis used for?
used to reduce the set of observed variables to a smaller, more parsimonious set of variables
what is the objective of an exploratory factor analysis?
the objective is to identify those items that most clearly represent the content domain of the
underlying construct
what does the confirmatory factor analysis assess?
Assesses the quality of the factor structure by statistically testing the significance of the overall
model.
which items should be eliminated?
Items that do not ‘load’ on specific factors are eliminated.
what should items of one scale be?
Items of one scale should be highly correlated
what should items of different sales be?
Items of different scales should be less correlated.
what should good items have?
large variance
what does internal consistency determine?
how well the items of a scale measure the same construct.
what is chronbachs alpha?
- Indicates the degree of interrelationship among the items of a scale (how strongly they are
correlated. - Items are deleted that impact the reliability of the scale negatively.
types of construct validity (2)
- convergent validity
- discriminant validity
what is convergent validity?
scores on a scales measuring a construct should relate to scores on other tests that are
theoretically related to the construct
what is discriminant validity?
scales measuring some construct should not be related to scores on other measures that are
theoretically unrelated to the construct
3 ways of replication
- Replicate the findings in a different sample.
- Alternatively one can also split the original sample and test the factor structure in the
two halves of the sample. - One can use a confirmatory factor analysis for replication.
