Project Prep Benchtest Flashcards
Research question characteristics
- Focused on a single problem
- Researchable using primary/secondary sources
- Feasible to answer within the timeframe and practical constraints
- Specific enough to answer thouroughly
- Complex enough to devlop the answer over a space of a paper or thesis
- Relevant to your field of study/or society more broadly
Types of Research questions
- Descriptive research
- Comparative research
- Correlational research
- Exploratory research
- Explanatory research
- Evaluation research
- Action research
What is in a problem statement?
- Context
- Specific issue being investigated
- Why this problem? Why now? Currency?
- Set objectives (project goals)
Descriptive research
What are the characteristics of X?
Comparative research
What are the differences and similarities between X and Y?
Correlational research
What is the relationship between variable X and variable Y?
Exploratory research
What are the main factors in X? What is the role of Y in Z?
Explanatory research
Does X have an effect on Y? What is the impact of Y on Z? What are the causes of X?
Evaluation research
What are the advntages and disadvantages of X? How well does Y work? How effective or desirable is Z?
Action research
How can X be acheived? What are the most effective strategies to improve Y?
S.M.A.R.T
Specific, Measurable, Attainable, Realistic, Timely
Inductive vs Deductive research
Developing a theory vs testing a theory
Exploratory vs Explanatory research
Exploring the main aspects of problem vs explaining causes and consequences of a well defined problem
Academic critique
- Deep dive into a single body of work
- Should be a counter argument - need to use external evidence and give counter points
Positivist
- Objective study
- Reductionist (break down complexities into simpler units of study)
- Verifying theories
- Can be studied in isolation
Critical Theorist
- Knowledge used to empower people
- Participatory
- Seeks to bring about change
- Focus on empowering groups
- Studied within that context
Constructivist
- Truth is relative to context
- Theory is open to interpretation
- Generates theories in a given context
- Cannot be studied in isolation
Pragmatist
- All research is biased
- No objective ‘truth’
- Works towards pratical solutions to problems
- Multiple answers
- Seek the best one(s)
Reliability
- How consistent are repeated measurements
- How close together are the measurements
Validity
Results correspond to the real thing
Types of reliability assessments
- Test-retest
- Inter-rater
- Internal
Types of validity assessments
- Construct
- Face
- Concurrent
- Predictive
Test-retest
-Determines reliability of the test and results over time
- Good indicator of reliability is strong correlation (r > 0.8) between same test given to same subjects over time
- Only works on consistent attributes
Inter-rater
- Determines reliability of test measurements and results gathered by different researchers
- Different people should give strongly correlated results
Internal
- Do you get same results if you use different tests to measure the same thing
- Strong correlation supports reliability
Construct (Validity assessment)
Does the test relate to high level theories
Face (Validity assessment)
Does test appear to test what it aims to test
Concurrent (Validity assessment)
- Does the test relate to an existing similar validated test
- Work is built on findings of another test and matches their work
Predictive (Validity assessment)
Does the test predict performance in a later developed test
Research Ethics
Concerns the responsibility of researchers to be honest and respectful to all individuals who are affected by their research studies or their reports of the studies’ results
Research integrity
Conducting research in a way that allows others to have trust and confidence in the methods used and findings that result from this
Bias
Conscious or unconscious influencing of the study and its results
Types of Bias
- Recall bias
- Selection bias
- Observation bias
- Confirmation bias
- Publishing bias
Recall Bias
- Survey respondents asked to recall events
- different types of events more likely to be remembered than others
Selections bias
Samples can sometimes under-represent certain people and over represent others
Observation bias
- Hawthorne Effect
- When participants are aware that they’re being observed they, either consciously or unconsciously, alter the way they act or the answers they give
Confirmation bias
- Occurs during interpretation of study data
- Researchers consciously or unconsciously look for information or patterns that confirm the ideas or opinions that they already hold
Publishing bias
- Studies with negative findings (nothing found) are less likely to be submitted by scientists or published by journals
- Perceived as less interesting
Avoiding bias
- Bias in per-course survey (unbalanced data) - automatic profiling
- Bias in learning about user instead of type of user (stereotyping) - different users in training and test sets
- Bias in future data predicting past - train on past, test on future
- Bias in unbalanced data sample - stratified sampling
Literature review
- A survey of scholarly sources on a specific topic(s)
- Provides an overview of current knowledge allowing you to identify relevant theories methods and gaps in existing research
Review article
- Summarises current state of understanding on a topic
- Surveys and summarises previously published studies - rather than report on new facts or analysis
- Gives roadmap on future research
- Can be used to back up the validity of your question
Surveys
Any method focused on asking Participants for responses
Purpose of Surveys
- Gather information not available from other sources
- Ubiased representation of population interest
- Collect information from many individuals to understand them as a whole
- Allows massive information gathering
Type of data collected by surveys
Mainly quantitative but qualitative methods can be used too
Pros of Surveys
- Can get info from large samples
- Can have different types and numbers of variables
- Gets info that’s hard to observe
- Easy and cheap
- Standardised stimulus - no observer subjectivity
Cons of surveys
- Intentional misreporting to hide inappropriate behaviour
- Poor recall
- Response rates are critical
- Can introduce bias from wording of questions
- Inflexible - can’t be changed during data gathering
- Not ideal for controversial issues
Survey Types by purpose
- Exploratory - form general ideas about research questions
- Descriptive - collect more specific descriptions of the variables of interest
- Explanatory - develop understanding of relationships among variables of interest
How can you validate surveys?
- Need to validate bias in question design
- Ask positive and negative questions - should be given opposite answers
- Validity of survey comes from the representativeness of the sample and the precision of the questions
- Face validity - Do questions appear reasonable and acquire data you want
- Content validity - Are questions all about issue and other subjects related to it
- Internal validity - Do questions imply the outcome you want to achieve
- External validity - Do questions elicit answers that are generalizable
Survey - Research questions
- Correlational questions
- Less technical questions - usability
- Exploratory questions
Types of sampling
- Random sampling- each member has equal chance of being picked
- Stratified sampling- use subsets of the population to sample - lower sampling error
- Systematic sampling- every Nth name is selected
- Quota sampling- researcher chooses necessary number of participants per stratum
- Purposive sampling- researcher selects participants according to criteria
Purpose of Observation
To understand how people naturally interact with products and people and the challenges they face
Pros of Observation
- Can get more subtle data
- Allows richly detailed description
- Viewing or participating in unscheduled events
- Improves quality of data collection
- Can see things you weren’t expecting
- Useful for formulating hypothesis
- Doesn’t depend on information provided by respondents
- Can deal infants/animals
Cons of Observation
- Less structured responses
- Get huge amount of data - analysing and not including bias is hard
- Difficult to replicate - lots of variables you don’t have control of
- Different researchers gain different understanding of what they observe
- Male/female researchers have access to different information
- Many events are uncertain in nature - difficult for researcher to determine time and place
- Can’t generalise
- Long and expensive
Observation - Research questions
- Exploratory
- Explanatory
Type of data collected by Observation
Typically qualitative but can be quantitative
Types of Observation
- Complete observer
- Observer as Participant
- Participant as Observer
- Complete Participant