L5, L6, L9, L1 Flashcards
What is Action research (AR)
Action research is described as a research method suitable for studying technology in social contexts. To only study real world problems without assisting or take part in the problem solving is regarded as unhelpful. Researcher helps client to identify and solve a problem
What is Technical action research (TAR)
Technical action research refers to a systematic approach where practitioners, often in technical fields such as engineering or computer science, engage in research activities to solve practical problems or improve processes within their professional domains. The researcher wants to learn something about a technique by
using it to solve a client’s problem
Contrasting TAR with AR in information system
AR in information systems
- Identify problem in an organization
- Jointly search for a solution and implement it
- Evaluate
- Specify learning
TAR is technology-driven, not problem driven!
- The technology is motivated by a desire to solve a class of problems
- Not a singular problem in a specific situation
What is hermeneutics?
Hermeneutics mean to develop the ability to understand phenomenon’s from another person’s perspective and to be able to understand and appreciate cultural and social forces that affects their situation.
What is an experminet?
A test under controlled conditions that is made to
demonstrate a known truth, examine the validity of a
hypothesis, or determine the efficacy of something previously
untried.
Experiments. When and why?
When we need to identify dependable relationships between a
cause and its effects. When we need to identify conditions under which such causal relationships hold. Experiments are important for many societal challenges across domains, like engineering (how flexible is this metal?)
Challenges of conducting
experiments
Choice of experimental design
Interpretation of the results
Statistics
Identifying and tackling validity threats
What is a cause?
Most causes are more accurately called inus conditions. Presence of a match → forest fire? Any given effect often requires many factors to occur, but we rarely know all of them, or how they relate to each other.
As a consequence, many causal relationships:
are probabilistic rather than deterministic in a strict sense
are not universal, but occur only under some conditions or in certain
contexts
Causality
It is rare that causal relationships hold under all conditions
with all types of people and at all historical time periods. It is vital to
identify relationships that hold more consistently.
Effect – what is an effect?
An effect can be understood as the difference between what actually happened (the fact) and what would have happened if a specific cause had not occurred (the counterfactual).
Effect: The effect is essentially the difference between what actually happened (fact) and what would have happened if the cause had not occurred (counterfactual).
What is a causal relationship?
The cause preceded the effect
The cause was related to the effect
We can find no plausible alternative explanation for the effect,
other than the cause
In experiments we:
Manipulate the presumed cause (and observe the outcome)
Observe whether variation in the cause is related to variation
in the effect
Attempt to rule out other explanations of the effect
Correlation does not imply causation
We may not know which variable came first (i.e., whether the presumed cause preceded the effect). We cannot rule out alternative
explanations. If there exists a third variable that causes the effect, this is called a confounding variable, or simply confound.
INDEPENDENT VARIABLE
Cause
This is the variable that is manipulated by the experimenter
The variable that i change
We want to know is if the independent variable affects the dependent variable in some way.
DEPENDENT VARIABLE
Effect (result of experiment)
This is the variable that is measured by the experiment
It depends on the independent variable
The dependent variable is the variable that is observed and measured to see if it is affected by changes in the independent variable.
Hypotheses
A hypothesis is a precise problem statement that can be
directly tested through an empirical investigation.
Example: “The iOS virtual keyboard is faster and more accurate
than the Android virtual keyboard.”
Randomization
A true experiment must have randomization. Random assignment is intended to create units that are probabilistically similar to each other on average
Example: Every individual has the same chance to end up in a given condition. Therefore, any observed differences in outcome are likely to be due to the treatment (= cause)
Simulations
Simulation refers to the process of creating and analyzing a
model or imitation of a real-world system or process to gain
insights into its behaviour or performance.
It involves using a computer program or mathematical model
to replicate a complex system’s essential features and
dynamics, allowing researchers to observe, experiment, and
make predictions in a controlled and virtual environment.
Key aspects of simulations
Modeling: Constructing a representation of the real-world system or
phenomenon using mathematical equations, algorithms, or computer programming.
Manipulation: Changing variables or parameters in the simulated
environment to observe their effects on the system’s behavior.
Analysis: Collecting and interpreting data generated by the simulation to draw conclusions about the real-world system.
Prediction: Extrapolating the simulation results to make predictions
about the future behavior or outcomes of the real-world system under different conditions.
Confounding variable
A variable that is not controlled but can affect the dependent
variable.
Control Variable
A control variable is a circumstance (not under investigation) that is kept constant while testing the effect of an independent variable
More control means the experiment is less generalizable (i.e., less applicable to other people and other situations)
Within-subjects
Each participant is tested on each condition
Within-subjects advantages:
Fewer participants
Less “variation due to participants”
No need to balance groups (because there is only one group!)
Between-subjects
Each participant is tested on one condition only
Between-subjects disadvantage
More participants (harder to recruit, schedule, etc.)
More “variation due to participants”
Need to balance groups (to ensure they are more or less the same)
Validity threats
Internal validity threats
* Reasons why inferences (assumption) that the relationship between two variables is causal may be incorrect
External validity
* Reasons why inferences about how study results would hold
over variations in persons, settings, treatments, and outcomes
might be incorrect.
What is SURVEYS
Any means of asking people for information
What are some Survey characteristics
With a paper or web-based survey, participants answer the questions on their own
- No chance to clarify or ask follow-ups
- It is hard to know how questions are interpreted
- Once the survey is out it is out
In what cases are Surveys used?
We use surveys when we need to investigate a clearly defined topic that involves measuring
- Knowledge
- Opinions
- Behaviours
- Perception
What are some survey development
We need our surveys to have a few important properties
- Relevant to our research questions
- Easy to interpret
- Be interpreted in the same way by all participants
general steps to survey development
- Define variables to measures
- Develop/adapt/find appropriate questions to measure those
- Validate and pilot
Variable definition and survey Questions
The first step in developing a survey is to define variables. Then, come up with questions to measure those variables. Often we use several questions for the same variable, to capture different aspects of the variable topic
These questions can be:
- Knowledge questions
- Likert scale questions
When developing questions there are some things to consider
- Clarity
- Consistency
- Question order
- Response order
Validation and Piloting
Validity concerns how well the survey measures what it intends to
measure
Expert review is one typical approach
- Focus on validity
- Ask domain experts about the survey questions (supervisor, peers)
Reliability concerns how consistently it does that
Distribute the survey to a small set of respondents and test how it goes
- Typically adding a comment question
- Focus on reliability (are people answering as expected
Survey Sampling
Our target group is called population - and we can typically not ask
everyone in that population. Instead, we need to draw a sample.
A sample is a subset of our population which we assume to be
representative of the population at large.
Random Sampling
Random sample
- We sample at random, and give everyone an equal chance of being selected
- Practically, we could select names at random from a list of everyone in the population
- Gold standard – but expensive and often infeasible
Stratified sampling
Stratified sampling (a practical approach to random sampling)
- The population is split into sub-population based on criteria such as age, gender….
- A random sample is drawn from each sub-population
- Intends to ensure a greater spread over the population
Convenience sampling
Convenience sampling
- Aka accidental sampling or opportunity sampling
- Involves recruiting participants “close at hand”
- Includes asking your friends, using social media, etc
- We cannot know how representative the sample is of the target population
Purposive sampling
Purposive sampling
- Selective sampling where “high-value” respondents are invited to answer
- The idea is to include participants who are likely to generate valuable data
- Useful in some cases, but hard to generalize from the results
Bias in Surveys
Sampling bias is skewness in results due to improper sampling.
Confirmation bias
- Participants tend to agree rather than disagree with opinions
- Try to avoid opinionated questions
Question order bias
- Question order may impact how participants answer
- Randomize questions to avoid
Survey fatigue bias
- Participants may get fatigued and stop answering properly
- Limit the survey length
How should an academic writing should be?
Clear:
- Easy to understand, impossible to misunderstand
Concise:
- Anything not needed for the task at hand should be omitted
Objective:
- Reflection of the work
Academic writing
Academic writing has a purpose (contribute to scientific knowledge), specific audience (scientists, academics), tone (unemotional, unbias)
What is the structure of academic writing?
Introduction
- Introduce the relevant literature
- Problem description
Materials and methods
- Introduce the study system
- What research methods have been used?
- How was the study done?
Results
- Objectively state finds
- Focus on results
Discussion
- Interpret you result
- Tie result back to the literature by answering the knowledge gap
Conclusions and implications
Bias in academic writing
Bias comes from you and your design, and it can affect the results based on several issues: e.g. chosen sample, investigation method and style of analysis.
What is Knowledge?
Knowledge is a fluid mix of framed experience, values, contextual information and expert insight that provides a framework for evaluating and incorporating new experience and information.
What is explicit knowledge?
Formal, codified: documents, databases, books etc
What is informal (tacit) knowledge?
Informal, uncodified: values, perspectives, culture, memories
What is categorical/descriptive knowledge?
categorical knowledge is the most fundamental type of knowledge, it is conceptualizing in terms of categories, concepts and relations
What is explanation knowledge/declarative knowledge?
Explanation knowledge is explanation of why a phenomenon is in a certain way.
Cause - effect - relationships
- if cause then effect
- if condition then result
What is evaluation/value knowledge?
Knowledge about the desirable, what we want given specific circumstances. Defining what “user friendly” means for example
What is normative/prescriptive knowledge?
Guiding knowledge
Advice, instructions, models, methods, tools, checklists… regarding a phenomenon.
What are some paradigms of research?
- Positivism – How reality is
- Realism – Objects exist independently of our will and knowledge
- Interpretivism – Understand humans in their role as social actors
- Pragmatism – If success then its good
What is positivism?
Working with an observable social reality, the researcher is independent of the research, highly structured.
What is realism?
Based on a belief that reality exists independent of human thoughts and beliefs. Social objects or phenomenon’s, external to or independent of individuals affect the way people perceive their world, whether they are aware of them or no
What is an artifact in IT-research?
An artifact is something that is created human begins which don’t exist without human involvement as something either by design or by interpretation.
Example of artifacts are computers, software, methods, models and so on.
