Research Design - Theories, Hypotheses And Variables Flashcards
theory
- A formal statement of relations among the observable phenomena.
- May contain hypothetical and unmeasurable constructs.
- Can give order to a body of scientific data.
Are not just merely descriptive but can be used to make testable predictions about unknown outcomes.
its a set of logically consistent statements about some phenonmenon that best summarises existing empirical knowledge of the phenomenon, organises this knowledge in the form of precise statements of relationships among variables ie laws, proposes sn explanation for then phenomenon and serves as the basis for making predictions. these predictions are then tested with research. theories in psychology differ in scope. some cover broad expanses of behaviour and are general theories. however more often a theory is focused on a specific aspect of behaviour. theories also differ in levels of precision some are strict in mathematical terms and others
described more simply as a set of logically connected statements.
an important feature of any theory is its continual evolution in light of new research. no theory is ever complete.
theories in psychology
- STM and LTM memory
- Attachment theory
- Cognitive dissonance theory
Triangle theory of love
hypothesis
- specific and way of testing a theory
- An attempt to organise certain data and specific relationships within a specific portion of a larger, more comprehensive theory.
- Generated from a theory.
- Can be precisely stated as a relationship between two measurable properties.
Can be falsifiable.
generate hypotheses from these theories
- Selective serotonin reuptake inhibitors (SSRIs) reduces depression…
- A lack of access to books in the home reduces child literacy…
- Listening to Mozart improves pupils’ learning…
Attending lectures improves grades…
null hypothesis
- What must be true if the hypothesis is not true.
- H1: SSRI treatment reduces BDI scores
- H0: SSRI treatment has no effect or increases BDI scores
- H1: Women have different IQs than men
H0: Women have the same IQs as men
directionality of hypothesis
- Directional hypothesis: one tailed
- Specifies a specific direction of effect
- Group A will have higher scores than Group B
- H1: A > B. H0: A ≤ B
- Non directional hypothesis: two tailed
- Direction of difference not specified
- Group A will have different scores than Group B
H1: A ≠ B. H0: A = B
a hypothesis leads an experiment
Experiments can provide evidence that a hypothesis is true.
* Typically, by showing the null hypothesis to be probably false.
* Experiments need variables…
What is a variable?
* Something that varies or can be varied!
* Must have at least two possible values.
* Must be observable and recordable.
Research is all about operationalized variables
operationalisation
- Each variable needs to be described clearly and unambiguously:
- A description of a construct such that another researcher can produce or measure the same thing.
- It is typically not a definition.
Eg An IQ test is not a definition of intelligence but it might be the operationalization of intelligence for the purpose of the experiment.
what do experimental psychologists do?
How does ______ affect _____?
e.g.:
How does smiling affect mood?
How does authority affect compliance?
How does inversion affect face recognition?
How does SSRIs affect depression scores?
How do hormones affect perception?
Independent variables
Dependent variables
Dependent variables
What we measure as outcomes (effects)
Dependent variable
* A response or behaviour that is measured that may be affected by changes to the independent variable.
* It needs to accurately reflect the performance being assessed.
It must show good variability over legitimate changes.
Independent variables
What we manipulate as predictors (causes)
variables must be
- Valid: That it measures what is supposed to be measured.
- How valid is recalling numbers strings a measure of cognitive function?
- The write up of a results should discuss what was measured (recalling digits) rather than what we hope we measured (cognitive function).
- Validity is a continuum not an absolute.
- Reliable: Produces consistent measurement in the same situations.
- Can be assessed by test-retest procedure
- Test of correlations between individual items
Cronbach’s Alpha - Multiple measurements are more reliable.
- Inter-rater reliability
Reliability is a continuum not an absolute. - test-retest and split-half
Becks depression inventory 21 four-option questions -
· Validity - good
· Good face validity
· Good correlation with clinical assessment through symptoms. (.66 - .76)
· BDI correlates with sense of humour loss
· BDI correlates with pessimism
· Reliability - good
· Test-retest reliability .90
· Cronbach’s Alpha .75 - .90
Over .7 is considered reliable
Hypothesis testing
- H1: A ≠ B. H0: A = B
- Tests whether we can reject the null hypothesis.
- Finds a p value. [Probability of the observed difference if the null hypothesis were true]
- p < .05 ‘accept’ the hypothesis
- Significant.
If stats are not significant (often p > .05) then we cannot reject H1 or H0.
More than one dependent variable
- With one IV and 100 different DVs (from 100 different cognitive tests) we are bound to find some ‘significant effects’.
- In fact, increasing the number of DVs means that it is harder to get a real significance effect (Bonferroni correction).
- Target p value (alpha) is divided by number of tests.
So if there are 5 tests done, p must be less than .05/5 which is .01.
significance and publishing the scientific literature
two conflicting theories - identify different predictions - construct a hypothesis that would be true for only one of the theories - operationalise the hypothesis into IVs and DVs - run the experiment to test the hypothesis
- nonsignificant - p > 0.05 - can’t distinguish between theories, don’t publish results
- significant - p < 0.05 - support one theory over the other, publish results
publication bias
significant results get published
non - significant ones do not
file-drawer effect
Meta- analysis
- Select all studies that looked at a particular topic.
- Published and unpublished
- Code the size of the effect found and the confidence of rejecting null.
Find the average effect and the confidence of rejecting the null given all of the data.
you find a topic, select the studies and assess overall effect
summary
- Theories lead to hypotheses which lead to experiments.
- Experiments need operationalised independent and dependent variables.
- Variables should be valid and reliable.
Other terms: Significance; Publication bias; Meta Analysis.
construct
a hypothetical factor that is not observed directly , its existence is inferred from certain behaviours and assumed to follow from certain circumstances.
the relationship between theory and research
the move from theory to research and back begins with the logical process of deduction, reasoning from a set of general statements towards the prediction of a specific event. deduction takes the form of the scientist reasoning that if the theory is correct then a specific reasearch outcome can be predicted and should occur with some probability greater than chance. The prediction about outcomes that is derived this way is called a hypothesis, which in general can be considered a reasonable prediction about a research result that should occur under certain circumstances. Hypotheses lead to the design of a study, which produces results as predicted or fails to produce them. In the former case, the theory is supported, and in the latter case it is not. If the theory is supported by a large number of research outcomes, a researcher’s confidence is high that the theory is a good one; to put it another way, we could say that inductive support for the theory increases when individual studies keep producing the results as predicted from the theory. Induction is the logical process of reasoning from specific events (the results of individual research studies) to the general (the theory).
theories being called into question
research project don’t always come out as expected. the study might not be a good test of hypothesis eg operational definitions of the variables might not be the best ones, it might have methodological flaws or it might fail for a reason that is never discovered. also measurements of psychological phenomena are imperfect so a failed experiment could be the result of measurement error. any study that fails to come out as it hopes it rarely calls a theory into question. if results repeatedly fail to support a theory however especially if they are conducted in different laboratories, confidence in the theory begins to lessen and it may be discarded or more likely altered.
psychologists don’t use the words prove and disprove as prove implies 100% truth and scientists can never be 100% sure their results are true.
theories
they may be supported and may be discarded but what happens most frequently is that they evolve as research accumulates and as challenges to the theory emerge. this happened in the case of cognitive dissonance.
attributes of good theories
- productivity - good theories advance knowledge by generating a great deal of research, an attribute that can be applied to dissonance theory.
- falsification
- parsimony
falsification
popper 1959 says science proceeds by setting up theories and then attempts to falsify them. theories that are continually resistant to falsification are accepted as possibly true. however this confidence is never absolute because of the limits of induction - future studies may require than a theory ne altered. poppers argument suggests that disconfirmation carries greater weight than confirmation. at the least, a stay that does not come out as expected requires that the disconfirmations be investigated throughly.
pseudoscience side steps falsification.
even though researchers might hope to find support for their own theories they are always trying to design experiments that can rule out one explanation or another.
parsimony
ideally studies include the minimum number of constructs and assumptions needed to explain the phenomenon adequately and predict future research outcomes. if two theories are equal in every way except one is more parsimonious then the simpler now is generally preferred.
the need for parsimonious explanations also guards against one of the social cognition biases - confirmation bias.
common misunderstandings about theory
theories are often misunderstood
developing research from other research
it occurs even when theory development is not the main focus. sometimes researches just want to investigate some phenomenon to discover regular, predictable relationships between variables and are not concerned about theory building. eg skinners operant conditioning
the most common sources of ideas for research in psychology are unanswered questions from studies just completed.
psychologists rarely conduct individual experiments that are separate from each other instead they build on programs of research, as series of interrelated studies. researchers often become involved in a specific area of investigation and complete a series of studies in that area that may last for years and may extend to many other researchers with interest in that topic. the conclusion of one project leads to the beginning of another as the experiment typically raises new questions. eg cognitive dissonance
this can be seen in any issue of a typical psychology journal.
research team and the ‘whats next’ question
researchers assemble research teams within their labs that operate under the apprenticeship model. the team typically includes a senior researcher, several graduate students who are working for the researcher and sometimes some highly motivated undergraduates who work under the direction of the graduate students who are the apprentices of the professor. they may have several experiments going on at once.
the team members use ‘whats next’ thinking so given the outcome of this study what should we do next? at some point someone will have an idea. if the idea is pursued, a procedure will be created, IRB approval will be sought and then tested in trial runs. ie pilot study, then revised or refined further and eventaully shaped into a study designed which is then completed then theres a new what’s next question.
a reserach study rarely stands by itself instead its outcome almost always leads to another study., often designed to clarify some unanswered questions of the first study or to extend the findings into a new direction. eg Roediger and Karpicke (2006) were curious about whether using a rereading study strategy or repeatedly testing students would lead to better memory for scientific texts. From this experiment, Roediger and Karpicke concluded that using testing as a study strategy leads to better memory in the long term compared to merely rereading texts. Based on these results, Roediger and Karpicke (2006) naturally adopted a “what’s‐next” style of thinking in their second experiment. Specifically, they wondered what would happen if students were given more than one opportunity to reread the texts or be repeatedly tested on the texts. They also were curious about how students thought they would do on the final memory test 1 week later, given the study strategies they used in the experiment.
pilot study
great way to to determine,ine whether the researchers are on the right track in developing sound procedures that will answer their empirical questions. The clarity of instructions to participants, the difficulty of the task that, the believability of a cover story (if the study involves deception), the duration of the experiment, and the adequacy of the materials are all important components of a sound research design that can be checked with a pilot study.
replication
refers to a study that duplicates some or all of the procedures of a prior study. there are two general types of replication.
a direct replication is an attempted reproduction of studies results testing the same type of sample and using the same exact procedures and statistical analysis as the original study. it is an attempted exact replication of a prior study usually done by a separate research team.
conceptual replication - parts of the procedure of a prior study are purposely changed in order to test predictions similar to those in the original study.
with successful replications researchers can be more confident that the results from their research are accurate and reliable. its also important in that if the results are accurate they can better inform public policy makers in developing public policy at the societal level. however failures to replicate can lead to less confidence in ones research and sometimes can lead to discovery of scientific fraud.
creative thinking in science
creative thinking in research design involves a process of recognising meaningful connections between apparently unrelated ideas and seeing those connections as the key to developing the study. this occurs in the context of some problem to be solved by a scientist with considerable knowledge of the problem.
pasteur ‘ chance favours the prepared mind’
serendipity does not itself produce an idea for a research study the serendipitous event must be seen by a scientist immersed in the topic as the missing piece that solves the problem at hand. this one reason why researches work in teams - the presence of several minds increases the chances someone will have an idea another team member will see as the missing piece to the puzzle.
a thorough knowledge of ones field may be prerequisite to creative thinking in science but such knowledge can also create rigid patterns of thinking that inhibit creativity. scientists occasionally become so accustomed to a particular method or so comfortable with a particular theory that they fail to consider alternatives thereby reducing the chances of making new discoveries.
reviewing the literature - computerised database searches
common ones are EBSCO, JSTOR. science direct and academic search premier and google scholar.
in psychology, the APAs psychINFO services provides a variety of electronic search tools. the primary database from which all of the others are derived is called psychINFO. PsycINFO. It includes references to journal articles, doctoral dissertations, technical reports, books, and book chapters. It includes journal articles dating back to 1887, the year American psychology’s first successful journal, the American Journal of Psychology, was founded. It includes more than 3 million abstracts, covers literature published in more than 2,500 journals, and is updated weekly. The features of PsycINFO are constantly being improved, so for the most up‐to‐date information, you should check out PsycINFO’s website at www.apa.org/psycinfo. Although PsycINFO is the primary database for psychologists, other databases may also be useful, depending on your research interests. For example, if you are interested in education and psychology, then ERIC may be a useful database; if you are interested in neuroscience, MedLine
in psychinfo use advanced search as can search by fields authors, article titles, journal name, and/or year of publication.
reviewing literature
The knowledge you gain from effectively reviewing the literature puts you in a better position to develop more ideas for research, formulate them as empirical questions, and develop them into testable hypotheses. With a good research hypothesis, you are ready to design a study that will provide answers to your empirical question.
essential features of experimental research
an experiment is a systematic research study in which the investigator directly varies some factor or factors holds all other factors constant and observes the results of the variation. the factors under control of the experimenter are the independent variables and the factors being held constant are the extraneous variables and the behaviours measured are dependent variables.
establishing independent variables
the factor of interest to the experimenter, the one being studies to see if it will influence behaviour. its sometimes called the manipulated variable or factor as the experimenter has complete control over it and creating the situations research participants will encounter during the study. they are also called manipulated independent variables. they must have a minimum of two levels - an experiment involves a comparison between two situations or conditions.
varieties of manipulated independent variables
situational variables - features in the environment that participants might encounter.
task variables - varying the type of task performed by subjects. can do this by giving participants different kinds of problems to solve.
instructional variables - manipulated by telling different groups to perform a particular task in different ways.
its possible to combine several types of indepedent variable in a single study. q
control groups
in some experiments the IV is whether or not some experimental condition occurs. some subjects get the treatment condition and others do not.
the term experimental group is used as a label for the situation in which the treatment is present. in the condition where treatment is withheld is called a control group. this satisfies Mill’s Joint Method.
the control group provides a baseline measure against which the experimental groups behaviour can be compared.
controlling extraneous variables
these are variables that are not of interest to the researcher but might influence the behaviour being studied if not controlled properly. as long as these are held constant they present no danger to the study.
if a researcher fails to control extraneous variables they can systematically influence the behaviour being measured. the result is called confounding. a confound is any uncontrolled extraneous variable that co-varies with the independent variable and could provide an alternative explanation of the results. a confounding variable changes in the same way that an independent variable changes and consequently its effect can’t be distinguished from the effect of the independent variable. when a study has a confound the resulst could be due to the effects of either the confounding variable or the IV or some combination of the two and there is no way to know which variable explains the results so results from studies with confounds are uninterpretable.
measuring dependent variables
its used to describe the behaviour that is the measured outcome of the study. the credibility of any experiment and its chances of discovering anything of value depend partly on the descisions made about what behaviours to measure and dependent variables. it can be hard to decide on DVs its good to know the prior research and use already-established dependent measures - those that have been shown to be valid and reliable. sometimes you have to develop a new measure however when you do a brief pilot study might help you avoid two major problems that can occur with poorly chosen DVs =
- ceiling effect - occurs when the average scores for the groups in the study are so high that no difference can be determined between conditions.
- floor effect - happens when all the scores are extremely low usually because the task is too difficult for everyone, so produces a failure to find any differences between conditions.
It is important to realize that a particular construct could be an independent, an extraneous, or a dependent variable, depending on the research problem at hand. An experimenter might manipulate a particular construct as an independent variable, try to control it as an extraneous factor, or measure it as a dependent variable.
subject variables
they are factors that are not directly manipulated by an experimenter. they are already existing characteristics of the individuals participating in the study eg gender, age, socioeconomic class, cultural group, intelligence, physical or psychiatric disorder and any personality attribute. when using subject variables in a study, the researcher can’t manipulate them directly but must select people for the conditions of the experiment by virtue of the characteristics they already have.
Some researchers, true to Woodworth’s original use of the term, prefer to reserve the term independent variable for variables directly manipulated by the experimenter. Others are willing to include subject variables as examples of a particular type of independent variable because the experimenter has some degree of control over them by virtue of the decisions involved in selecting them in the first place and because the statistical analyses will be the same in both cases.
only a study using manipulated IVs can be called an experiment in the strictest sense of the term. - it is sometimes called a true experiment.
studies using IVs that are subject variables are sometimes called ex post facto studies, natural groups studies or quasi experiments. studies often include both manipulated and subject independent variables.
subject variables effect the kinds of conclusion that can be drawn.
drawing conclusions when using subject variables
with subject variables causal conclusions can’t be drawn as the experimenter holds less control. the experimenter can vary a factor but cannot hold all else constant. Selecting participants who are high or low on anxiety proneness does not guarantee the two groups will be equivalent in other ways. In fact, they might differ in several ways (e.g., self‐confidence, tendency to be depressed) that could influence the outcome of the study. When a difference between the groups occurs in this type of study, we cannot say the differences were caused solely by the subject variable. In terms of the conditions for causality, although we can say the independent variable precedes the dependent variable, we cannot eliminate alternative explanations for the relationship because certain extraneous factors cannot be controlled. When subject variables are present, all we can say is that the groups performed differently on the dependent measure.
