Final Deck Flashcards
What is Action research (AR)
Action research is described as a research method suitable for studying technology in social contexts. Researchers help clients to identify and solve a problems.
ETHICS IN RESEARCH
Ethics in research concerns issues relating to the research content and the researcher’s relationship to the task. Research ethics include aspects that follow from mandatory legislation, but research ethics are broader than this.
Honesty in terms of ethics
meaning scrutinizing the research. Researchers should conduct their work with honesty and integrity. This includes reporting findings in an open, fair and objective way. Honesty also extends to acknowledging limitations and uncertainties in the research process and results.
Respect when it comes to ethics
Respect for research participants, society, and the environment. This involves obtaining informed consent from participants, protecting their confidentiality and privacy, and considering the potential impact of the research on individuals and communities.
Accountability
Researchers are responsible for the entire research process, from idea to publication and beyond. This means taking ownership of their work, ensuring it adheres to ethical standards and regulatory requirements, and being prepared to address any consequences or implications that may arise.
Transparency when it comes to ethics
Transparency is essential for ensuring the reproducibility and credibility of research findings. Researchers should clearly document and communicate their methods, data collection procedures, and analysis to enable others to assess the validity and reliability of the research.
Positive results bias
Researchers must guard against the tendency to selectively report or emphasize positive outcomes while neglecting negative findings. This positive result bias can skew the overall body of evidence and mislead both researchers and the wider community. It’s important to present all findings.
When is a Qualitative method used?
A qualitative method is used when answering “why” questions. For example people’s perceptions, opinions, ideas, interpretations, values, behavior
When is a Quantitative method used?
A Quantitative method is used when answering “how many/much” questions.
Explain what a Qualitative method is?
Qualitative method is suitable for researching “why” questions, such as why people have a specific opinion. In a qualitative method data is interpreted, categories and structured. This data is often observations, textual and figures. The qualitative approach is to observe and interpret. A qualitative analysis is thematic analysis (identifying. Analyze and interpret pattern)
Explain what a Quantitative method is?
A quantitative method is suitable when trying to research “how many/ much” questions. In a quantitative method data is measured, counted and ranked. This data is often Numerical and statistics. The quantitative approach is to measure and test. The analysis method that is used is statistics. In a quantitative method variables and categories are predefined.
Give examples of Qualitative methods
Case study
Focus groups
Interviews
Diary study
Ethnography
Card sorting
Explain what a Case study is?
Case study is a qualitative research method that entails conducting a comprehensive examination of a specific case, which could pertain to an individual, a group, or an organisation. It is used to gain a detailed understanding of a specific case or cases. Case studies often take a holistic approach, considering multiple aspects of the case rather than focusing on just one variable or factor.
Explain what an interview is in the context of a Qualitative method?
An interview is a qualitative research method that relies on asking questions in order to collect data. Interviewing people is at the heart of qualitative research.
There are several types of interviews:
* Structured interviews have predetermined questions asked in a predetermined order.
* Unstructured interviews are more free-flowing.
* Semi-structured interviews fall in between.
Explain what a focus group is?
A focus group is a research method used to gather data through group interaction. A small number of carefully chosen individuals debate a certain issue in a group. focus groups identifies and examines people’s thoughts and behaviours.
What is Technical action research (TAR)
Technical action research refers to a systematic approach where practitioners, often in technical fields such as computer science, engage in research activities to solve practical problems or improve processes within their professional domains. The researcher wants to learn something about a technique by using it to solve a client’s problem
Contrasting TAR with AR in information system
AR in information systems
- Identify problem in an organization
- Jointly search for a solution and implement it
- Evaluate
- Specify learning
TAR is technology-driven, not problem driven!
- The technology is motivated by a desire to solve a class of problems
- Not a singular problem in a specific situation
In action research the researcher wants to solve a problem for a client, in technical action research the problem is only a way to learn about a specific technique
In technical action research, what are the three roles of the researcher?
Designer, helper, researcher
Design Science Research (DSR
Design Science Research (DSR) is a research methodology used primarily in information systems. It focuses on creating innovative artifacts to solve complex problems or improve existing systems.
What is an artifact in IT-research?
An artifact is something that is created by human begins which don’t exist without human involvement as something either by design or by interpretation.
Example of artifacts are computers, software, methods, models and so on.
What is hermeneutics?
Hermeneutics means to develop the ability to understand phenomenon’s from another person’s perspective and to be able to understand and appreciate cultural and social forces that affects their situation.
What are the 3 types of literature review and there purpose?
Simplified literature review
- Identify and summarize seminal work in your chosen topic
Systematic literature mapping
- To structure a research area
Systematic literature review
- Gathering and synthesizing evidence
Describe the literature review process?
The literature review process involves systematically searching and critically evaluating existing literature relevant to a specific research topic.
Some steps that make a good literature review are:
Clearly defining scope and purpose: Determine the specific research topic you want to explore and establish the boundaries of your review (e.g., timeframe, geographical scope, types of sources).
Search for Relevant Literature: Use academic databases, library catalogs, and search engines to identify relevant literature. Employ a combination of Search terms, keywords, and strings, Quality criteria, Assessment and ranking functions.
Organize and Synthesize Information: Organize the selected sources chronologically based on common themes, topics, or key concepts. Identify patterns, trends, and gaps in the literature, and synthesize the findings to develop a coherent narrative that addresses your research question.
Write the Literature Review: Write the literature review using a clear and structured format. The review should be well organized and replicable.
Some steps that make bad literature review are:
Lack of Focus: A bad literature review may lack a clear focus or research question, resulting in a broad and unfocused review of the literature. Without a specific focus, the review may cover irrelevant topics, leading to confusion and lack of coherence.
Poor Organization: An ineffective literature review may suffer from poor organization, with sources arranged in a careless manner. This can make it difficult for readers to follow the text or identify key themes and trends within the literature.
Lack of Depth: A shallow literature review may provide only superficial coverage of the existing literature, failing to delve deeply into key concepts, theories, or empirical findings. This lack of depth can undermine the credibility and rigor of the review.
Lack of Critical Analysis: A weak literature review may lack critical analysis or evaluation of the sources reviewed. Instead of critically assessing the strengths, weaknesses, and limitations of each source, the review may simply summarize or describe the findings without offering meaningful insights or interpretations.
Existence of Previous Reviews: If comprehensive literature reviews already exist in the field, conducting another review without adding new perspectives or insights may be redundant and unnecessary.
What two stages of interpretation are important to hermeneutics?
1.Uncovering interpretation (finding how others have categorised the world)
2.Assigning interpretation (creating new categories)
What is Knowledge?
Knowledge is a fluid mix of framed experience, values, contextual information and expert insight that provides a framework for evaluating and incorporating new experience and information.
What is explicit knowledge?
Formal, codified: documents, databases, books etc
What is informal (tacit) knowledge?
Informal, uncodified: values, perspectives, culture, memories
What is categorical/descriptive knowledge?
It is conceptualizing in terms of categories, concepts and relations
What is explanation knowledge/declarative knowledge?
Declarative knowledge is explanation of why a phenomenon is in a certain way.
Cause - effect - relationships
- if cause then effect
- if condition then result
What is evaluation/value knowledge?
Knowledge about the desirable, what we want given specific circumstances.
Defining what “user friendly” means for example
What is normative/prescriptive knowledge?
Guiding knowledge
Advice, instructions, models, methods, tools, checklists… regarding a phenomenon.
What are some paradigms of research?
- Positivism – How reality is
- Realism – Objects exist independently of our will and knowledge
- Interpretivism – Understand humans in their role as social actors
- Pragmatism – If success then its good
What is positivism?
Working with an observable social reality, the researcher is independent of the research, highly structured.
What is realism?
Based on a belief that reality exists independent of human thoughts and beliefs. Social objects or phenomenon’s, external to or independent of individuals affect the way people perceive their world, whether they are aware of them or no
What is INFORMATION SYSTEMS (IS)
The discipline of information systems (IS) focuses on information
(data in a specific context) together with information capturing,
storage, processing and analysis/interpretation in ways that supports
decision making.
COMPUTER SCIENCE (CS)
Computer science is the study of computers and algorithms,
including their principles, hardware and software design, their
applications, and their impact on industry and society
INFORMATION TECHNOLOGY (IT)
Information technology (IT) is the study of systemic approaches to select, develop, apply, integrate, and administer secure computing technologies to enable users to accomplish their personal, organizational, and societal goals
Why do we do Litterature Review?
- To examine scientific progress and know what has been done
- To find a research gap in order to motivate new research
How do we do Litterature Review?
By identifying and summarizing existing research and results
What is Iterative process?
Returning to earlier stages as you search, read an evaluate
What are some steps in the Literature search process?
- Formulate a research topic and useful keywords
- Refine the search. ex: date limits
- Review your results and revise your search
- Evaluate your results
- Keep up-to-date with new literature on your topic
What should the literature review results be summarized in?
- A related work section
- Identification of research gap/problem description
What three types of litterature reviews are there?
- Simplified literature review
- Systematic literature review
- Systematic literature mapping
What type of literature review is used to provide a survey of the field?
Systematic literature mapping/scoping
At what stage should the simplified literature review be conducted?
Before planning any form of research
What is the aim of the simplified literature review?
Idetify and summarize influential work in your chosen topic
Purpose of Systematic literature mapping?
To structure a research area
Purpose of systematic literature review?
Gathering and synthesizing evidence
What type of literature review creates categories?
Systematic literature mapping
What type of literature review is used for evaluation and interpretation of research relevant to a particular topic?
Systematic Literature Review
What are the three high-level phases in a Systematic Literature Review?
- Planning the review
- Conducting the review
- Reporting the review
What are individual studies that contribute to a systematic review called?
Primary studies
What is a form of a secondary study?
Systematic Review
What 6 types of information should be in a Literature Review Protocol?
- Search terms, keywords and strings
- Databases and sources used
- Inclution criteria
- General and specific exclution criteria
- Quality criteria
- Assessment and ranking functions
What are some signs of a bad literature review?
- Lack of focus
- Poor organizaton
- Poor quality of sources
- Not enough information in the protocol
What types of bias can there be in literature review?
- Language bias, geographical
- Selection bias
- Selecting only results that support your hypothesis
- Time related bias: Outdated information
What is an experminet?
A test under controlled conditions that is made to demonstrate a known truth, examine the validity of a hypothesis, or determine the efficacy of something previously untried.
When and why would an experiments be conducted?
When we need to identify dependable relationships between a cause and its effects. When we need to identify conditions under which such causal relationships hold. Experiments are important for many societal challenges across domains, like engineering (how flexible is this metal?)
Challenges of conducting
experiments
Choice of experimental design. Interpretation of the results Statistics
Identifying and tackling validity threats
What is a cause?
Most causes are more accurately called inus conditions. Presence of a match → forest fire? Any given effect often requires many factors to occur, but we rarely know all of them, or how they relate to each other.
As a consequence, many causal relationships: are probabilistic rather than deterministic in a strict sense
are not universal, but occur only under some conditions or in certain
contexts
Causality
It is rare that causal relationships hold under all conditions with all types of people and at all historical time periods. It is vital to identify relationships that hold more consistently.
Effect – what is an effect?
An effect can be understood as the difference between what actually happened (the fact) and what would have happened if a specific cause had not occurred (the counterfactual).
Effect: The effect is essentially the difference between what actually happened (fact) and what would have happened if the cause had not occurred (counterfactual).
What is a causal relationship?
The cause preceded the effect. The cause was related to the effect. We can find no plausible alternative explanation for the effect, other than the cause
In experiments we: Manipulate the presumed cause (and observe the outcome) Observe whether variation in the cause is related to variation
in the effect Attempt to rule out other explanations of the effect
Correlation does not imply causation
We may not know which variable came first (i.e., whether the presumed cause preceded the effect). We cannot rule out alternative explanations. If there exists a third variable that causes the effect, this is called a confounding variable, or simply confound.
INDEPENDENT VARIABLE
This is the variable that is manipulated by the experimenter the variable that can change
We want to know if the independent variable affects the dependent variable in some way.
DEPENDENT VARIABLE
This is the variable that is measured by the experiment It depends on the independent variable. Effect (result of experiment).
The dependent variable is the variable that is observed and measured to see if it is affected by changes in the independent variable.
Hypotheses
A hypothesis is a precise problem statement that can be directly tested through an empirical investigation.
Example: “The iOS virtual keyboard is faster and more accurate
than the Android virtual keyboard.”
Randomization
A true experiment must have randomization. Random assignment is intended to create units that are probabilistically similar to each other on average
Example: Every individual has the same chance to end up in a given condition. Therefore, any observed differences in outcome are likely to be due to the treatment (= cause)
Simulations
Simulation refers to the process of creating and analyzing a model of a real-world system to gain insights into its behaviour or performance.
It involves using a computer program to replicate a complex system’s essential features and dynamics, allowing researchers to observe, experiment, and make predictions in a controlled and virtual environment.
Key aspects of simulations
Modeling: Constructing a representation of the real-world system orphenomenon using mathematical equations, algorithms, or computer programming.
Manipulation: Changing variables or parameters in the simulated
environment to observe their effects on the system’s behavior.
Analysis: Collecting and interpreting data generated by the simulation to draw conclusions about the real-world system.
Prediction: Extrapolating the simulation results to make predictionsabout the future behavior or outcomes of the real-world system under different conditions.
Confounding variable
A variable that is not controlled but can affect the dependent variable.
Control Variable
A control variable is a circumstance (not under investigation) that is kept constant while testing the effect of an independent variable
More control means the experiment is less generalizable (i.e., less applicable to other people and other situations)
Within-subjects
Each participant in a survey is tested on each condition/question
Within-subjects advantages:
Fewer participants
Less “variation due to participants”
No need to balance groups (because there is only one group!)
Between-subjects
Each participant in a survey is tested on one condition/question only
Between-subjects disadvantage
More participants (harder to recruit, schedule, etc.)
More “variation due to participants”
Need to balance groups (to ensure they are more or less the same)
Validity threats in research
Validity threats refer to factors that can compromise the validity of the research findings.
Internal validity threats ( researcher’s ability)
- Reasons why inferences (assumption) that the relationship between two variables is causal may be incorrect
External validity
- Reasons why inferences about how study results would hold
over variations in persons, settings, treatments, and outcomes
might be incorrect.
What is SURVEYS
Any means of asking people for information
What are some Survey characteristics
Surveys can be a paper or web-based, participants answer the questions on their own
- No chance to clarify or ask follow-ups
- It is hard to know how questions are interpreted
- Once the survey is out it is out
In what cases are Surveys used?
We use surveys when we need to investigate a clearly defined topic that involves measuring
- Knowledge
- Opinions
- Behaviours
- Perception
What to think about when developing a survey?
We need our surveys to have a few important properties
- Relevant to our research questions
- Easy to interpret
- Be interpreted in the same way by all participants
general steps to survey development
- Define variables to measures
- Develop/adapt/find appropriate questions to measure those
- Validate and pilot
Variable definition and survey Questions
The first step in developing a survey is to define variables. Then, come up with questions to measure those variables. Often we use several questions for the same variable, to capture different aspects of the variable topic
These questions can be:
- Knowledge questions
- Likert scale questions
When developing questions there are some things to consider
- Clarity
- Consistency
- Question order
- Response order
Validation and Piloting
Validity concerns how well the survey measures what it intends to
measure
Expert review is one typical approach
- Focus on validity
- Ask domain experts about the survey questions (supervisor, peers)
Reliability concerns how consistently it does that
Distribute the survey to a small set of respondents and test how it goes
- Typically adding a comment question
- Focus on reliability (are people answering as expected
Survey Sampling
Our target group is called population - and we can typically not ask everyone in that population. Instead, we need to draw a sample. A sample is a subset of our population which we assume to be representative of the population at large.
Random Sampling
Random sample
- We sample at random, and give everyone an equal chance of being selected
- Practically, we could select names at random from a list of everyone in the population
- Gold standard – but expensive and often infeasible
Stratified sampling
Stratified sampling (a practical approach to random sampling)
- The population is split into sub-population based on criteria such as age, gender….
- A random sample is drawn from each sub-population
- Intends to ensure a greater spread over the population
Convenience sampling
Convenience sampling
- Aka accidental sampling or opportunity sampling
- Involves recruiting participants “close at hand”
- Includes asking your friends, using social media, etc
- We cannot know how representative the sample is of the target population
Purposive sampling
Purposive sampling
- Selective sampling where “high-value” respondents are invited to answer
- The idea is to include participants who are likely to generate valuable data
- Useful in some cases, but hard to generalize from the results
Bias in Surveys
Sampling bias is skewness in results due to improper sampling.
Confirmation bias
- Participants tend to agree rather than disagree with opinions
- Try to avoid opinionated questions
Question order bias
- Question order may impact how participants answer
- Randomize questions to avoid
Survey fatigue bias
- Participants may get fatigued and stop answering properly
- Limit the survey length
In Data Analysis researcher may impact the results. True/False?
True
Traits of Qualitative Data (3)
- Non-numerical information
- Often textual
- Focuses on thematic analysis
Traits of Quantative Data(3)
- Numerical
- Best described in numbers
- Uses statistics as a method
What is a confidence interval? (CI)
Measure of how confident we are in our results
What is a good confidece level?
95%
What typ of statistics are used to describe data with means, modes, standard deviation and such?
Descriptive Statistics
What type of statistics are used to draw conclutions, correlations, hypothesis testing?
Infernal Statistics
What are the 4 data levels?
- Nominal
- Ordinal
- Interval
- Ratio
What type of data is Nominal Data?
Categorical data. Ex: Which animal is your favourite?
What descriptives does Nominal Data use?
Frequencies and percentages
What type of data is Ordinal Data?
Ordered data with undefined or unequal lengt between steps. Ex: How tasy is vanilla ice cream? - Tasty, - Neutral, - Untasty
What descriptives does Ordinal Data use?
Frequencies, percetages and median calculation
What type of data is Interval and Ratio?
Ordered data with equal distance between data points
What is the difference between Interval and Ratio data?
-Interval: no defined 0
-Ration: defined 0
What is Standard Deviation?
A measure of how dispersed (spread of data) the data is
Inferential Statistics is a good way to use a smaller group to make an assumtion for the population at large? True/False?
True
What significance level(SI) should Inferential Statistics have?
95%
What are the 2 main approaches to Inferential Statistics?
- Hypothesis testing
- Correlation testing
What is a null hypotheses?
Opposite variable of a hypotheses.
What would the null hypothesis be of “Everybody likes ice cream”?
“We cannot prove that everybody likes ice cream”
What is a Type 1 Error in Statistical Errors?
Reject a true null hypothesis (false positive)
What is a Type 2 Error in Statistical Errors?
Accept a false null hypothesis (false negative)
What tests can be used to get the p-value?
- T-test
- Rank-Sum test
- Chi2
What is linear correlation?
Correlation between 2 variables where one variable increses with the other
What is p-hacking?
-Shotgunning statistical results until something is positive
- It involves selectively analyzing data or conducting multiple statistical tests until a desired result
- Something will be positive sooner or later
What is Thematic Analysis
Systematically generating codes and themes from qualitative data
What are the 6 steps of Thematic Analysis?
- Familiarizing with the data
- Generating initial codes
- Searching for themes
- Reviewing themes
- Defining and naming themes
- Producing report
What is an example of a Thematic Analysis?
Ex: Inteviews with ice cream manufacurers about how their changes to recepies might change peoples opinions on favourite ice cream flavours.
How should an academic writing be?
Clear:
- Easy to understand, impossible to misunderstand
Concise:
- Anything not needed for the task at hand should be omitted
Objective:
- Reflection of the work
Academic writing
Academic writing has a purpose (contribute to scientific knowledge), and it’s for specific audience (scientists, academics), tone (unemotional, unbias)
What is the structure of academic writing?
Introduction
- Introduce the relevant literature
- Problem description
Materials and methods
- What research methods have been used?
- How was the study done?
Results
- Objectively state finds
- Focus on results
Discussion
- Interpret you result
- Tie result back to the literature by answering the knowledge gap
Conclusions and implications
Bias in academic writing
Bias comes from you and your design, and it can affect the results based on several issues: e.g. chosen sample, investigation method and style of analysis.
What are some qualitative methods?
- Case study research
- Ethnography
-Interviews
-Focus group
-Diary study
What are the main principles of a qualitative method?
- To answer “why” questions.
- To observe and interpret data
Describe a case study research process.
The purpose of a case study research is to conduct a comprehensive examination of a specific case to understand the complexities of the case.
What is ethnography?
Ethnography is a method that looks at social interaction of users in a specific environment. Data is collected through observation and interviews to conclude how individuals and societies function.
Describe a focus group.
A focus group is used to gather data through group interactions. A small number of people debate a certain topic to shed light on certain questions. They can provide more nuanced feedback and are easier to to organize than experiments.
Describe a diary study.
In a diary study, participants keep a log of their thoughts, experiences and activities over a period of time. Diary studies provide a self-reported record of the participants behavior which can be used to understand habits.
Describe participant observation.
This is a type of study that can help us understand how participants act and behave. A researcher enters a specific environment and observes how the individuals interact and behave.
Describe card sorting.
Card sorting is a way to understand how individuals group and categories certain information. It can give a picture of an individuals thought process.
Describe a longitudinal study.
This type of study is conducted over a longer period of time with repeated observations. A disadvantage is that they are usually expensive and time-consuming.
