Subjectivity, Objectivity & Bias In Welfare Measurements Flashcards
Define subjective
Subjective = An individual’s perception of reality
What does subjectivity deal with?
Your reality
Your judgement
What you perceive to be really happening
Define objectivity
Objectivity = reality can be defined and agreed on
Absolute truth which we can empirically prove/see in the same way
Everyone perceives it in the same way
Define anthropomorphism
Anthropomorphism = the attribution of human characteristics or behaviour to a god, animal or object
What is anthropomorphism an element of?
Perspective taking
What is perspective taking?
Perspective taking = being able to understand another humans perspective and empathise with their experience
What has perspective taking allowed humans to be successful in?
Social species
Cooperation
How has perspective taking also been argued to be an adaptive ability?
Being able to guess what the animal is thinking made it easier to hunt and domesticate animals
What is Watanabe 2007’s definition of anthropomorphism in psychology?
The explanation of animal minds through introspection of researchers
What does Watanabe 2007’s definition mean?
Anthropomorphism is an internal reflection about what we think the animals internal/mental experience is
This means that it’s not the animals actual mindset but the researchers subjective take on what the animal is actually thinking
There is no common object of research among researchers so this violates objectivity
What are the 3 problems with anthropomorphism in welfare?
1) We might be wrong about the animals internal experience
2) Humans might be better at accurately assessing function than feelings
3) Introspective nature of anthropomorphism means it doesn’t convince others with different perspectives q
Give an example where “we might be wrong about the animals internal experience”
Clever Hans
Hans was a horse that was thought to be able to count
People would tell Hans a number and he’d count to that number by pawing at the ground
A psychologist then did a study on Hans where they noticed that Hans was actually paying close attention to his owners micro-movements in his expression and body movements
When Hans was given a number his owner would tense up nervously
Hans would then start counting till his owner relaxed at the correct number
So in assuming Hans could count the truth could’ve been ignored
Why can anthropomorphism between humans be hard and what does this mean for applying anthropomorphism to animals?
Even between humans anthropomorphism can be hard due to behavioural expression varying
So when applying anthropomorphism to animals it can be really hard
Prey species are much less likely to express emotional states than predators, especially if it shows weakness
What are one of the methods humans use to process huge amounts of information rapidly?
One of the ways in which humans can reduce the cognitive effort associated with processing information is by using ‘short cut cues’ or ‘heuristic rules’
What are heuristic rules?
A commonsense rule (or set of rules) intended to increase the probability of solving some problem
How do heuristic rules make anthropomorphism hard?
We may have learned assumptions in daily human-human interactions which are not applicable to animals but we use them as a reflex anyways
The reason this is bad is because animals see and perceive the world differently to humans due to different perceptual abilities
This is because animals have evolved to fit different habitats and so have different priorities to humans
Due to how bad heuristic rules can be when anthropomorphising what can be argued about species less similar to humans?
As a result of heuristic rules it can be argued that species less similar to humans are harder to anthropomorphise due to:
- different anatomies
- behavioural differences
- what we perceive to be complexity
Why might we be wrong in our anthropomorphising?
It’s hard to determine emotions in other humans
Humans tend to use ‘short cut cues’/’heuristic rules’ which may not apply to animals
Animals perceptual abilities and biological properties differ from human ones
Humans can’t justify for other species
Name and describe a study that shows humans might be better at accurately assessing function than feelings
Lynn Snow et al 2005
In the experiment children and caretakers were asked to rate the impact of a chronic physical condition of pediatric patients in terms of:
- health and physical ability
- emotional states
The children and caretakers rated the health and physical aspects of the condition as the same
The caretakers also underestimated the emotional impacts of the condition
This shows human proxies rate human physical systems more accurately than internal experiences
What does “introspective nature means it doesn’t convince others with different perspectives” mean for anthropomorphism?
Anthropomorphism is a single individuals judgement, meaning other people may or may not believe it
Anthropomorphism when applied to animal welfare can potentially be dangerous in what 2 different ways? And who came up with these problems?
Meagher 2009
1) Unthinkingly assuming animals think and feel as humans do
2) Blindly denying the possibility of a shared attribute
What is educated anthropomorphism?
Educated anthropomorphism = being conscious about the potential impacts of different sensory perceptions, biological properties and the limitations associated with anthropomorphism
What 3 ways can educated anthropomorphism be useful?
1) Alerting us that an animal might be suffering or hold untested cognitive abilities
2) Fueling our compassion so we care enough to act
- Butterfield et al 2012
- Anthropomorphism makes us care about an animals welfare even if sentience hasn’t been scientifically proven
3) Generating hypothesises that we can scientifically test
- However, anthropomorphism is not measuring welfare or can it be used as evidence to study the hypothesis
Define objectivity
A characteristic of scientific practice expressing the idea that bias, personal interests or values should not influence methods or results
What do social scientists argue about objectivity?
Social scientists argue that objectivity is impossible
How is objectivity somewhat aspirational?
Objectivity is somewhat aspirational in that there are degrees of objectivity and absolute objectivity
What do you need from an objective method of research and why?
Data collection is purely quantitative
Quantitative data allows the use of statistical analysis to test probabilities that the data findings represent a common ‘truth’ for the majority of the population
Give some examples of objective measures
Number of grooming bouts
Number of heart beats per minute
Concentrations of faecal cortisol
Time spent in location A vs B
What makes a measure objective?
They’re external to our judgement
Is welfare an objective concept?
No it’s subjective since it’s based on an animals internal experience
Since we can’t know what an animals internal experience is directly what do we use?
Welfare indicators that we hypothesise will measure the animals internal experience indirectly
Are the welfare indicators used subjective or objective?
It depends on the indicator since the indicators we use can be subjective or objective
Why are measures of welfare called indicators?
Measures of welfare are called indicators because welfare can only be indicated not directly measured
Name a completely subjective indicator and how it’s used
Snap judgment indicator
This is where we look at an animal and decide it’s emotional state
Because it’s an internal judgement by you it can’t be shared with others in the same way/understanding
Describe a method of subjective welfare assessment
Another method is by creating a subjective assessment where you quantify welfare
This is where you rate the animals situation based on your own judgement
This is subjective because a person can only see the judgement but not be able to link it to the animal at the time it was made
They can only register the person’s judgement of the animals mental state but can’t agree on that judgement or make a different one
What does an objective measure need?
An objective measurement involves a separation between the measurement stage and the judgement stage of what that measure means
This allows everyone to look at the measurement and potentially make a different judgement or agree with a judgement about what that data means
This therefore is more robust in terms of evidence
Subjectivity can arise at various different levels when we’re considering measuring welfare. Where/when are the 4 ways subjectivity can occur?
1) During rapid judgement
2) As an intrinsic quality of what you are trying to measure
3)In your measurement techniques
You’ll need to decide a threshold which is a judgement you’ll make making it subjective
This is true even if the measurement technique is objective
4) In your interpretation of the results
This is still subjective even if the results are objective
Name a study that shows bias in objective recordings
Tuyttens et al 2014
What happened in Tuyttens et al 2014’s paper?
Tuyttens et al showed students pictures or videos of pigs
They told the students 1 video was of pigs from a higher social breeding value and that those pigs where associated with better growth in their pen mates than the other pig video
They then got the students to objectively record the frequency of positive and negative pig interactions from both videos of the pigs
They also gave the students an ethogram so they knew what to record
However both videos of the pigs were the same
What were Tuyttens et al 2014s results?
The students reported significantly less positive social behaviour in the control pigs than they pigs said to have higher social breeding value
The opposite was noted for the negative social behaviour
What was the cause of Tuyttens et al 2014s results?
The difference in scores for the same video was caused by the students anticipating a certain outcome for each video they saw and so had a bias toward positive or negative scoring for each video
This meant that the students created an expectation bias which affected the results that they got
Does expectation bias occur with behaviour
No
When can expectation bias occur?
When interpretation of any measure involves human judgement
Give examples of when bias can occur
Experimental design
Selection of subjects
Interpretation of results
Assertions regarding implications of what the data means
Name an example of a experiment that shows bias
Bennett et al 2009 F-mri fish study
What happened in Bennett et al’s 2009 F-MRI Fish study?
Bennett et al 2009 were looking at studying perspective taking in humans
To do this Bennett et al 2009 would show a group of human subjects pictures of other humans with emotional expressions
They’d then ask the human subjects what the facial expressions meant while they were attached to a F-MRI scanner
This allowed the researchers to know which brain parts activated while the human subjects were making there judgement
To calibrate the F-MRI scanner the researchers placed a dead Atlantic salmon inside the scanner and showed it human facial expressions
The researchers measured brain activity by recording oxidised levels of voxels
Surprisingly when the fish was shown human facial expressions some areas of the fishes brain lit up
Why did Bennett et al 2009’s fish show brain activity?
Within the study they were doing lots of statistical tests on huge data sets without correcting for multiple testing
The more tests you do the more likely it is you’ll get a false positive
This is why the dead salmon showed brain activity
What does Bennett et al 2009’s fish F-MRI study tell us?
It shows that we can falsely record positives if we don’t control for various aspects in our experimental design
What did Lehrner 2002 do?
Lehrner 2002 lists alternative sources of bias in data collection that are relevant to behaviour data
These sources of bias can also be relevant to collections of other types of data
What are the 4 relevant sources of bias in data collection that Lehrner 2002 lists?
1) Presence of observer cues of subject
2) Physical arrangement of the study animal and/or observer can introduce error where position makes behaviour difficult to observe
3) Observer error can be caused by inexperience and poorly defined categories including “drift” movement of observer over time from some base point
4) Error of recording
What does Lehrner 2002 mean by “presence of observer cues of subject”?
If the animal that we are studying can see, hear or smell us it can change the animals behaviour and potentially also alter their physiology
This means our results will be altered due to us being near the animal
What does Lehrner 2002 say will happen when “physical arrangement of the study animal and/or observer can introduce error where position makes behaviour difficult to observe”?
This generates a likelihood of bias in recording whether or not the behaviour happened or confusing the behaviour with a different behaviour
This problem can also be relevant to collecting physiological data
An example of this is if a heart monitor is incorrectly placed on the animal leading to incorrect data collection
What does Lehrner 2002 mean by “observer error can be caused by inexperience and poorly defined categories including ‘drift’ movement of observer over time from some base point”?
This means that over time there could be a change in how things are recorded, which introduces problems with our data
Give examples of how “error of recording” could take place according to Lehrner 2002?
Error of recording can be due to:
- poor technique
- poor equipment
- mental lapses
- inexperience
If we practice poor science, or there’s lots of bias, and we don’t correct for it/interpret the data with that knowledge then our findings may not be correct. What can this lead to?
1) A situation may be considered a welfare issue when it is not
This then leads to:
Time and money being wasted on addressing it
Time and money not used on actual welfare issues
2) A true welfare issue may not be resolved and may continue to be ignored
If you have bias or practice poor science but still get correct findings they may not inspire confidence in the welfare issue which can lead to what?
Damaged reputation Low career prospects Low likelihood of further research in the topic Low public confidence Delays in resolving the welfare problem
What are the 3 R’s?
Replacement
Reduction
Refinement
What are the 3 R’s for?
The principals of the 3R’s were developed over 50 years ago providing a framework for performing more humane animal research
If we practice poor science, and even if our findings are correct, nobody will believe us which means what?
This means the subjects would have been used for nothing
The study will also be a waste of the funder’s investment, with the money being better spent on another study
What are the 7 things in the good scientific practice criteria?
1) A clear statement of hypothesis and selection of appropriate approaches to test it
2) Recognising risks of bias
3) Careful study design to minimise bias
4) Standardisation and clear description of methods
5) Discussing possible sources of bias and study limitations in publication
6) Explore alternative explanations for the findings
7) Implementation of the 3 R’s
Reduction
Refinement
Replacement
Why is “standardisation and clear description of methods” part of the good scientific practice criteria?
This allows other people to:
See if potential problems were introduced
Replicate the study if necessary
Give some examples of different observer based methods
Scales/rating/scoring systems
Questionnaires
Qualitative Behavioural Assessment
What’s the 2 most important things when using observer based methods and what do they mean?
Precision = able to consistently get the same/very similar value Accuracy = validity
Accuracy is the most important thing you want
Explain how you can use scores for an observer rating?
You give scores for a variable you’re trying to measure using units of measurements that were defined by the researcher
The units are not standard metrics used normally like Kg or seconds
In a questionnaire the variable is also known as an item
What can you use observer rating scores to estimate?
Either a:
Directly observable constructs/features
A subjective construct/feature
What can you use observer rating scores to estimate?
Either a:
Directly observable constructs/features
A subjective construct/feature
Because observer rating scores are subjective what does this mean for the results?
The results of observer rating scores are subjective so they can be influenced by experience and personal views
Name examples of observer ratings that are based on 1 scale and explain them
Likert/numeric scales
A Likert scale will have a neutral point like this
Strongly agree _1_2_3_4_56 Strongly disagree
However Likert scales can be altered to be uni-directional like with the dog example below
These scales are categories used to judge what best describes the situation
The person making the score will select which category best represents the feature they’re trying to measure
An example is a dog ages 8yeara old will fall into the 5-10 year old category
Give an example and explain how visual analogue scales can be used as observer ratings
😁 no pain 😁________________ 😫worst pain ever😫
The person making the judgement will then mark on the line where they feel it best suits the situation between the 2 ends or the category descriptors at the ends of the line
The researcher will then measure from the starting point to the point of the cross on the line
Visual analogue scales are usually 100mm long
Then the researcher can have an accurate descriptor
For example the X was at 16mm compared to 100mm
The items or scales can be combined into what?
Multi-item instruments
What can multi-item instruments be used to look at?
Multi-item instruments can be used to look at the combination of either:
- several different behaviours
- several different construct attributes
What is the multi-item instrument called if it measures a single construct?
Index
What is the multi-item instrument called if the items used are predicted to relate to eachother?
Scale
What do you call a multi-item instrument if it will be used by someone other than the researcher?
Questionnaire
Qualitative Behavioural Assessments are also known as what?
QBA’s
Why are Qualitative Behavioural Assessments controversial approaches to studying animal welfare?
QBA’s are completely subjective
What does Wemelsfelder 1997 argue?
1) Traditional animal behaviour methods cannot capture subjective states, such as the need flow of behaviour over time and across situations
2) We should consider subjective experiences as a perspective
What does Wemelsfelder 1997 mean when she says “traditional animal behaviour methods cannot capture dynamic subjective states, such as the need flow of behaviour over time and across situations”?
Wemelsfelder 1997 argues that objective methods cause us to miss important information which humans can naturally understand by looking at dynamic changes over time in interactions with the environment
The animals dynamic changes over time and its interactions with the environment are an expression by the animal which reveals its mental state and perspective
What does Wemelsfelder 1997 mean when she says “we should consider subjective experiences as a perspective”?
The behaviour and subjective experience form an expressive whole reflected in the way the “subject interacts with and pays attention to a situation”
What does Wemelsfelder 1997 propose about Qualitative Behavioural Assessments?
Wemelsfelder 1997 proposed that Qualitative Behavioural Assessments assessed how the subject interacts with and pays attention to a situation, and that’s associated with welfare while incorporating elements of observer ratings
What was the original Qualitative Behavioural Assessment called?
Free choice profiling
How does Free Choice Profiling work?
First a large group of observers watched the animal live or on a recording
Then each observer would write down adjectives they felt best described the animals expression in the situation
The group would then take those adjectives and rewatch the video or a recording of the event they just saw and score them using visual analogue scales based on how well they believe it described the situation
They’d then use Procrustes analysis and Principal Components analysis to see if it was statistically significant or not
Name a Qualitative Behavioural Assessment that was created in 2013
Fixed trait items welfare quality assessment
Who created the Fixed Trait Items Welfare Quality assessment and why?
Around 2013 the Welfare Equality Project was developing welfare assessment protocols for a range of farm species
They were looking for a way to assess positive welfare since at the time there were no validated methods to do so
Qualitative Behavioural Assessments were suggested as an option but it needed to be more standardised
How did the Welfare Equality Project 2013 standardise Qualitative Behavioural Assessments?
They’d usually only use a few observers
They’d ask the observers to discuss terms and reach a consensus on the meaning of those terms
Then the observers worked from a fixed list of terms to rate/score behaviour on training videos to achieve consistent responses
The observers then used the fixed terms to rate and score the animal subjects in the same way as the free choice profiling method
They’d then use the Principal Components analysis to kinda group datasets together
Name an example of Free Choice Profiling
Wemelsfelder et al 2000
Explain what happened in Wemelsfelder et al 2000’s experiment
Wemelsfelder et al 2000 got 2 groups of pigs, 10 pigs houses in an enriched environment and 10 pigs in an unenriched environment.
18 observers then scored individual experimentally isolated pig interactions with a familiar human in a separate test pen from the pigs housing
The 18 observers then scored the interactions using Free Choice Profiling
The data was then analysed so that the adjectives can be combined into axes/profiles that explain variation in the data
Wemelsfelder et al 2000 then took 2 profiles which explained most of the data variation and plots them on a table against each other
Then they looked at how much consensus there was between the combined axeses
In the experiment the observers significantly agreed on the pigs mental state
They then compared the profiles to what would happen if the observers randomly attributed adjectives
Wemelsfelder et al 2000 then looked at different observers judgements to see how well their judgement of the pigs matched the consensus profile
They then used the adjectives plotted on a graph to describe the axeses generated by the profiles
The researchers would look at their observers results and try to assess generally what the axeses mean
In this experiment only a few people would say the same adjective, however they assumed that different adjectives meant the same thing
Then Wemelsfelder et al 2000 would plot individual pigs on the consensus profile graph made and see how well the pigs matched the profiles dimensions
They’d then look to see if the 2 groups of pigs were actually separated along 1 or both of the axeses
What were Wemelsfelder et al 2000’s findings?
Wemelsfelder et al 2000 saw no difference between the 2 pig groups in their interactions with a familiar human present
What are the 4 limitations specific to Free Choice Profiling Qualitative Behavioural Assessments?
1) Free choice profiling uses lay human terms that are relevant to human emotional states, but not necessarily biologically appropriate for superficially similar states in animals
2) Free Choice Profiling doesn’t test assumptions that different observers mean the same thing with different terms
3) Free Choice Profiling tells us if the observers agree about their judgements but there is no valid test of the animals state
They’re precise about the judgement but potentially inaccurate
4) Free Choice Profiling requires prior understanding of what the specific terms mean in relation to the specific context
What was the Procrustes analysis originally developed for and what does this mean for Qualitative Behavioural Assessments?
The Procrustes analysis was originally developed to look at consensus between different assessors of food quality
They were looking at features of a specific objective item
However in Qualitative Behavioural Assessments Procrustes analysis is used for a consensus on a subjective experience by another individual
As a result there is potential for bias
This is a more specific criticism of Free Choice Profiling’s validity
What are some criticisms of Qualitative Behavioural Assessments?
1) The Procrustes analysis was originally designed for looking at features of a specific objective item not subjective experiences of another individual
As a result there’s potential for bias
2) many studies indicate correlation of Qualitative Behavioural Assessments with other indicators such as quantified behaviour and physiological measures
However these studies have been critized with bias through experimental design
- they may use the same people to measure the QBA and to measure the behaviour leading to an expectation bias
- some of these studies can be critised on the basis of bias or non-independence
3) the studies that use Qualitative Behavioural Assessments and show validity are usually studies where the animal was under acute challenge rather than a chronic state
4) It is also unclear if Qualitative Behavioural Assessments validity varies between species
5) some studies in the literature do still show no agreement between QBA and other indicators
However these studies have been critised for not using enough observers or using untrained observers
6) more recent work suggests that Qualitative Behavioural Assessments:
Are reliable in lab/experimental work
Not reliable in field situations
What are the 6 advantages of observer rating methods?
1) They can be rapid and efficient
2) They do not require costly or delicate equipment
3) They are generally non-invasive
4) No extensive labour/time is generally needed
5) They can take advantage of familiarity with animals
6) They can allow integration of information over time so we don’t need to keep sampling
What are the 6 limitations of observer ratings?
1) They may be more suitable for some species than others
2) They may be more suitable to certain traits or behaviours than other ones that are less easier to monitor or notice
3) We may not recognise the importance of some responses
4) They are subjective and so are subject to bias
5) Ratings are assigned based on observer norms and experience and so can change over time
6) Therefore we cannot assume subjective measures represent the whole picture and should be used in conjunction with other measures to ensure better validity
What happened in Tuyttens et al’s 2014 study when they used a Qualitative Behavioural Assessments?
Tuyttens et al 2014 got the same students from previous tests to do a Qualitative Behavioural Assessment on videos of hens
Tuyttens et al 2014 gave the students fixed descriptors rated on visual analogue scores
The students were told that 1 video was of organic hens and 1 video was of commercial hen aviary system
However unknown to the students they were the same video again
When the students did the QBA the data showed a shift in ratings for 13 of the 19 traits looked at
This therefore shows expectation bias is also seen in the subjective scoring of Qualitative Behavioural Assessments
What are the 6 way for reducing subjectivity and bias in observer ratings?
1) We need well defined, unambiguous criteria for scoring
2) Training in recognising traits or constructs and their range of expressions
3) Sufficient sampling
4) Agreement of multiple observers
5) Blinding observers to treatments to minimise expectation bias
6) we want to use validated scoring systems whenever possible
