Chapter 2: Research Tools Flashcards
What do Research Resources consist of?
1) Library & Resources
2) Techniques of measurement
3) Statistics, computers & languages.
What can Measurement do? (4)
- Provide limits to data
- Measure both substantial and insubstantial data
- Examine relationships mathematically
- Set acceptable qualitative / quantitative standards
Measurement can generally be considered as a ___________.
Comparison
Define Measurement:
It is a thing or concept measured against a point of limitation.
What are the 6 types of Validity in Measurement?
1) Face or Subjective Judgement
2) Criterion
3) Content
4) Construct
5) Internal
6) External
Define Face Validity or Subjective Judgement:
The degree to which a procedure appears effective in terms of its stated aims (IQ)
Define Criterion Validity:
Two measures of validity where the second checks the accuracy of the first and become a sort of standard. (SAT, MCAT)
When is Content Validity used?
Where data is unsubstantial.
Provide an example of Content Validity:
Does the exam ask questions about material that was taught.
Define Construct Validity:
How well a test measures the concept it was designed to study, any concept that cannot be measured / observed easily
- Honesty
Define Internal Validity:
Making certain that a change in the dependent variable is the result of independent variable, and not how you measured it.
Define External Validity:
A process where conclusions are drawn from your sample may be generalized / extrapolated.
Provide an example of an External Validity:
Can the methodology and results be repeated, transferable, and replicable by others.
What is the bottom line in relating Measurement and Validity?
Is the measurement we are making actually measuring what we think we are measuring?
We use _______ to measure outcomes and draw inferences about the study subjects based on the ________ of our measurements.
Tools, interpretations
It is important to measure the _____________ things.
Obvious (Ex. Horizontal v. Vertical lines)
What can measurements be considered as?
A comparison
What are the types of measurements?
- Nominal Level
- Ordinal Level
- Interval Level
- Ratio Level
Define the Nominal Level of measurement:
One object is different from another
Define the Ordinal Level of Measurement:
One object is bigger / better than the other.
What is an example of a Nominal Level of Measurement?
- Assign a name to items or groups
- Divide data into discrete categories (%)
What is an example of an Ordinal Level of Measurement?
- Example: Elementary, high school, college, etc.
Define Interval Level Measurement:
Equal units of measurement and a zero point established. arbitrarily assigned values.
What is an example of an Interval Level Measurement?
- Degrees
- cm, m, etc.
Define Ratio Level Measurement:
One object is so many times taller/heavier/brighter.
Is Ratio Data and Interval Data quantitative or qualitative?
Quantitative
Is Ordinal Data and Nominal Data quantitative or qualitative?
Qualitative
What are some ways we would know to use a Nominal Measurement?
If one object is different from another.
How would we know to use Ordinal Measurement?
If one object is bigger or better or more of anything than the other.
How would we know to use an Interval level Measurement?
If one object has so many units (cm, m, etc.) more than another
How would we know to use a Ratio Level Measurement?
If one object is so many times as big or bright or as tall as another.
What do we depend on from the measurement instrument to collect data we need?
Soundness and effectiveness
What are the 6 types of validity in measurement?
- Face Validity or Judgement
- Criterion Validity
- Content Validity
- Construct Validity
- Internal Validity
- External Validity
What is Face Validity or judgement?
Does our measurement tool record what it is supposed to and is our sample representative (can be a subjective judgement call)
What is Criterion Validity?
Two measurements, with the first being confirmed by the second, or one predict the other.
What is an example of Criterion Validity?
SAT vs GPA vs. MCAT
What is Content Validity?
Whether the measurement is valid in getting the information needed.
Provide an example of Content Validity:
Final exam of a course to assess learning - should cover all topics that was taught in the class and not include anything that was not covered.
What is Construct Validity?
For concepts that cannot be measured or observed easily (honesty, intelligence) but can be measured by observing other indicators that are associated with it.
- Valuable in social sciences, where there is a lot of subjectivity to concepts.
What is Internal Validity?
Change in dependent variable is truly a result of change in the independent variable.
What is External Validity?
The extent to which you can generalize the findings of a study to other situations, people, settings and measures.
- In other words, can you apply the findings of your study to a broader context?
What does it mean to have a reliable research tool?
How accurate is the research tool you’re using to measure your variable of interest.
____________ is simply a tool of research for looking at data from different angles.
Statistics
What tool can we use to help us lessen the degree of uncertainty in our understanding of relationships and causes we investigate.
Statistics
What needs to be clearly stated when developing the research proposal and executing research activities?
Goals and objectives
What type of thinking is required when resolving a problem & deriving new meaning about the problem by interpreting the results of your research efforts?
Analytical Thinking
What are six key things to do when formulating a research problem?
- Use the least amount of words
- Use the exact word
- Use short words instead of long words as much as possible
- Ensure a logical, step-wise connection between ideas
- Discard superfluous words
- Be prepared to modify
Define a Subproblem:
Small questions and challenges leading to your main question.
How can we minimize subproblems?
Define terms of the research focus and state assumptions around the topic in question.
What can we use to target a question?
Hypothesis
Define a Null Hypothesis:
Any apparent relationship you
find is due to chance (it is not real).
When do we use a Null Hypothesis?
We use the null hypothesis because statistical analysis tests the validity of the null hypothesis
Define an Alternate Hypothesis:
There is a predictive pattern or relationship between variables that is not due to chance
When do we accept a null hypothesis?
We accept the null hypothesis unless definite evidence of a pattern/relationship exists and we significantly minimize our speculation on what the relationship or outcome we found means.
What are Multiple Hypotheses?
When you have competing hypotheses in your study and you can have combinations that are true and false.
True or False: You may need more than one hypothesis to confirm your findings.
True.
______________: We call these sub-hypothesis.
When you need more than one hypothesis to confirm your findings.
When is a sub-hypothesis addressed in your study?
In the Problem Statement and the Introduction.
How do we test a hypothesis?
- Directly measure the variables of interest
- Assess if the struggle surrogate variable(s) are strong indicators of a key variable that you cannot measure
Define a Surrogate Variable:
Indirect or related variables when we cannot directly measure key variable.
What kind of results can we get?
- Positive
- Negative
- None at all
IF your study is well designed, scientifically valid and well implemented the outcome will be useful as long as your hypothesis is __________.
Defensible.
What are 6 benefits to running Pilot / Preliminary study?
- Validate question
- Practice study techniques
- Standardize approach to data collection.
- Add data to study
- Estimating study size
- Statistical analysis & change to practice some analysis on a small set of data.
What are the two types of Data Collection Approaches?
- Observational
- Manipulative
Define a Correlation Study:
A type of research design that looks at the relationships between two or more variables
Define an Observational / Correlation Approaches:
Simply observing the focus variable (E.g., Tail feathers of long tailed birds)
What kind of variation does Observational Studies use?
Natural Variation
Define Manipulative Approaches:
Using artificial variation to examine data.
True or False: Using both Observational and Manipulative Data Collection in a study is better than simply using one approach.
True
Why is it important to be aware of Reverse Causation?
Because the association between two variables can be the reverse than you expect.
If we cannot directly measure key variables then our indirect or related variables are called ____________.
surrogate variables
Define Content Validity:
It evaluates how well an instrument (like a test) covers all relevant parts of the construct it aims to measure
What does Validity determine?
That the results are reflective of the thing you are interested in.
Provide an example of Reverse Causation:
Instead of X causing Y, as is the case for traditional causation, Y causes X
What does the Problem Statement Identify?
It is the broad overarching issue, it does not specify any variables generally speaking.
