Subjectivity, Objectivity & Bias In Welfare Measurements

Question 1

Define subjective

Answer 1

Subjective = An individual’s perception of reality

Question 2

What does subjectivity deal with?

Answer 2

Your reality
Your judgement
What you perceive to be really happening

Question 3

Define objectivity

Answer 3

Objectivity = reality can be defined and agreed on
Absolute truth which we can empirically prove/see in the same way
Everyone perceives it in the same way

Question 4

Define anthropomorphism

Answer 4

Anthropomorphism = the attribution of human characteristics or behaviour to a god, animal or object

Question 5

What is anthropomorphism an element of?

Answer 5

Perspective taking

Question 6

What is perspective taking?

Answer 6

Perspective taking = being able to understand another humans perspective and empathise with their experience

Question 7

What has perspective taking allowed humans to be successful in?

Answer 7

Social species

Cooperation

Question 8

How has perspective taking also been argued to be an adaptive ability?

Answer 8

Being able to guess what the animal is thinking made it easier to hunt and domesticate animals

Question 9

What is Watanabe 2007’s definition of anthropomorphism in psychology?

Answer 9

The explanation of animal minds through introspection of researchers

Question 10

What does Watanabe 2007’s definition mean?

Answer 10

Anthropomorphism is an internal reflection about what we think the animals internal/mental experience is
This means that it’s not the animals actual mindset but the researchers subjective take on what the animal is actually thinking
There is no common object of research among researchers so this violates objectivity

Question 11

What are the 3 problems with anthropomorphism in welfare?

Answer 11

1) We might be wrong about the animals internal experience
2) Humans might be better at accurately assessing function than feelings
3) Introspective nature of anthropomorphism means it doesn’t convince others with different perspectives q

Question 12

Give an example where “we might be wrong about the animals internal experience”

Answer 12

Clever Hans
Hans was a horse that was thought to be able to count
People would tell Hans a number and he’d count to that number by pawing at the ground
A psychologist then did a study on Hans where they noticed that Hans was actually paying close attention to his owners micro-movements in his expression and body movements
When Hans was given a number his owner would tense up nervously
Hans would then start counting till his owner relaxed at the correct number
So in assuming Hans could count the truth could’ve been ignored

Question 13

Why can anthropomorphism between humans be hard and what does this mean for applying anthropomorphism to animals?

Answer 13

Even between humans anthropomorphism can be hard due to behavioural expression varying
So when applying anthropomorphism to animals it can be really hard
Prey species are much less likely to express emotional states than predators, especially if it shows weakness

Question 14

What are one of the methods humans use to process huge amounts of information rapidly?

Answer 14

One of the ways in which humans can reduce the cognitive effort associated with processing information is by using ‘short cut cues’ or ‘heuristic rules’

Question 15

What are heuristic rules?

Answer 15

A commonsense rule (or set of rules) intended to increase the probability of solving some problem

Question 16

How do heuristic rules make anthropomorphism hard?

Answer 16

We may have learned assumptions in daily human-human interactions which are not applicable to animals but we use them as a reflex anyways
The reason this is bad is because animals see and perceive the world differently to humans due to different perceptual abilities
This is because animals have evolved to fit different habitats and so have different priorities to humans

Question 17

Due to how bad heuristic rules can be when anthropomorphising what can be argued about species less similar to humans?

Answer 17

As a result of heuristic rules it can be argued that species less similar to humans are harder to anthropomorphise due to:

      - different anatomies
      - behavioural differences
      - what we perceive to be complexity

Question 18

Why might we be wrong in our anthropomorphising?

Answer 18

It’s hard to determine emotions in other humans
Humans tend to use ‘short cut cues’/’heuristic rules’ which may not apply to animals
Animals perceptual abilities and biological properties differ from human ones
Humans can’t justify for other species

Question 19

Name and describe a study that shows humans might be better at accurately assessing function than feelings

Answer 19

Lynn Snow et al 2005
In the experiment children and caretakers were asked to rate the impact of a chronic physical condition of pediatric patients in terms of:
- health and physical ability
- emotional states
The children and caretakers rated the health and physical aspects of the condition as the same
The caretakers also underestimated the emotional impacts of the condition
This shows human proxies rate human physical systems more accurately than internal experiences

Question 20

What does “introspective nature means it doesn’t convince others with different perspectives” mean for anthropomorphism?

Answer 20

Anthropomorphism is a single individuals judgement, meaning other people may or may not believe it

Question 21

Anthropomorphism when applied to animal welfare can potentially be dangerous in what 2 different ways? And who came up with these problems?

Answer 21

Meagher 2009
1) Unthinkingly assuming animals think and feel as humans do

2) Blindly denying the possibility of a shared attribute

Question 22

What is educated anthropomorphism?

Answer 22

Educated anthropomorphism = being conscious about the potential impacts of different sensory perceptions, biological properties and the limitations associated with anthropomorphism

Question 23

What 3 ways can educated anthropomorphism be useful?

Answer 23

1) Alerting us that an animal might be suffering or hold untested cognitive abilities

2) Fueling our compassion so we care enough to act
- Butterfield et al 2012
- Anthropomorphism makes us care about an animals welfare even if sentience hasn’t been scientifically proven

3) Generating hypothesises that we can scientifically test
- However, anthropomorphism is not measuring welfare or can it be used as evidence to study the hypothesis

Question 24

Define objectivity

Answer 24

A characteristic of scientific practice expressing the idea that bias, personal interests or values should not influence methods or results

Question 25

What do social scientists argue about objectivity?

Answer 25

Social scientists argue that objectivity is impossible

Question 26

How is objectivity somewhat aspirational?

Answer 26

Objectivity is somewhat aspirational in that there are degrees of objectivity and absolute objectivity

Question 27

What do you need from an objective method of research and why?

Answer 27

Data collection is purely quantitative
Quantitative data allows the use of statistical analysis to test probabilities that the data findings represent a common ‘truth’ for the majority of the population

Question 28

Give some examples of objective measures

Answer 28

Number of grooming bouts
Number of heart beats per minute
Concentrations of faecal cortisol
Time spent in location A vs B

Question 29

What makes a measure objective?

Answer 29

They’re external to our judgement

Question 30

Is welfare an objective concept?

Answer 30

No it’s subjective since it’s based on an animals internal experience

Question 31

Since we can’t know what an animals internal experience is directly what do we use?

Answer 31

Welfare indicators that we hypothesise will measure the animals internal experience indirectly

Question 32

Are the welfare indicators used subjective or objective?

Answer 32

It depends on the indicator since the indicators we use can be subjective or objective

Question 33

Why are measures of welfare called indicators?

Answer 33

Measures of welfare are called indicators because welfare can only be indicated not directly measured

Question 34

Name a completely subjective indicator and how it’s used

Answer 34

Snap judgment indicator
This is where we look at an animal and decide it’s emotional state
Because it’s an internal judgement by you it can’t be shared with others in the same way/understanding

Question 35

Describe a method of subjective welfare assessment

Answer 35

Another method is by creating a subjective assessment where you quantify welfare
This is where you rate the animals situation based on your own judgement
This is subjective because a person can only see the judgement but not be able to link it to the animal at the time it was made
They can only register the person’s judgement of the animals mental state but can’t agree on that judgement or make a different one

Question 36

What does an objective measure need?

Answer 36

An objective measurement involves a separation between the measurement stage and the judgement stage of what that measure means
This allows everyone to look at the measurement and potentially make a different judgement or agree with a judgement about what that data means
This therefore is more robust in terms of evidence

Question 37

Subjectivity can arise at various different levels when we’re considering measuring welfare. Where/when are the 4 ways subjectivity can occur?

Answer 37

1) During rapid judgement
2) As an intrinsic quality of what you are trying to measure

3)In your measurement techniques
You’ll need to decide a threshold which is a judgement you’ll make making it subjective
This is true even if the measurement technique is objective

4) In your interpretation of the results
This is still subjective even if the results are objective

Question 38

Name a study that shows bias in objective recordings

Answer 38

Tuyttens et al 2014

Question 39

What happened in Tuyttens et al 2014’s paper?

Answer 39

Tuyttens et al showed students pictures or videos of pigs
They told the students 1 video was of pigs from a higher social breeding value and that those pigs where associated with better growth in their pen mates than the other pig video
They then got the students to objectively record the frequency of positive and negative pig interactions from both videos of the pigs
They also gave the students an ethogram so they knew what to record
However both videos of the pigs were the same

Question 40

What were Tuyttens et al 2014s results?

Answer 40

The students reported significantly less positive social behaviour in the control pigs than they pigs said to have higher social breeding value
The opposite was noted for the negative social behaviour

Question 41

What was the cause of Tuyttens et al 2014s results?

Answer 41

The difference in scores for the same video was caused by the students anticipating a certain outcome for each video they saw and so had a bias toward positive or negative scoring for each video
This meant that the students created an expectation bias which affected the results that they got

Question 42

Does expectation bias occur with behaviour

Answer 42

No

Question 43

When can expectation bias occur?

Answer 43

When interpretation of any measure involves human judgement

Question 44

Give examples of when bias can occur

Answer 44

Experimental design
Selection of subjects
Interpretation of results
Assertions regarding implications of what the data means

Question 45

Name an example of a experiment that shows bias

Answer 45

Bennett et al 2009 F-mri fish study

Question 46

What happened in Bennett et al’s 2009 F-MRI Fish study?

Answer 46

Bennett et al 2009 were looking at studying perspective taking in humans
To do this Bennett et al 2009 would show a group of human subjects pictures of other humans with emotional expressions
They’d then ask the human subjects what the facial expressions meant while they were attached to a F-MRI scanner
This allowed the researchers to know which brain parts activated while the human subjects were making there judgement

To calibrate the F-MRI scanner the researchers placed a dead Atlantic salmon inside the scanner and showed it human facial expressions
The researchers measured brain activity by recording oxidised levels of voxels
Surprisingly when the fish was shown human facial expressions some areas of the fishes brain lit up

Question 47

Why did Bennett et al 2009’s fish show brain activity?

Answer 47

Within the study they were doing lots of statistical tests on huge data sets without correcting for multiple testing
The more tests you do the more likely it is you’ll get a false positive
This is why the dead salmon showed brain activity

Question 48

What does Bennett et al 2009’s fish F-MRI study tell us?

Answer 48

It shows that we can falsely record positives if we don’t control for various aspects in our experimental design

Question 49

What did Lehrner 2002 do?

Answer 49

Lehrner 2002 lists alternative sources of bias in data collection that are relevant to behaviour data
These sources of bias can also be relevant to collections of other types of data

Question 50

What are the 4 relevant sources of bias in data collection that Lehrner 2002 lists?

Answer 50

1) Presence of observer cues of subject
2) Physical arrangement of the study animal and/or observer can introduce error where position makes behaviour difficult to observe
3) Observer error can be caused by inexperience and poorly defined categories including “drift” movement of observer over time from some base point
4) Error of recording

Question 51

What does Lehrner 2002 mean by “presence of observer cues of subject”?

Answer 51

If the animal that we are studying can see, hear or smell us it can change the animals behaviour and potentially also alter their physiology
This means our results will be altered due to us being near the animal

Question 52

What does Lehrner 2002 say will happen when “physical arrangement of the study animal and/or observer can introduce error where position makes behaviour difficult to observe”?

Answer 52

This generates a likelihood of bias in recording whether or not the behaviour happened or confusing the behaviour with a different behaviour
This problem can also be relevant to collecting physiological data
An example of this is if a heart monitor is incorrectly placed on the animal leading to incorrect data collection

Question 53

What does Lehrner 2002 mean by “observer error can be caused by inexperience and poorly defined categories including ‘drift’ movement of observer over time from some base point”?

Answer 53

This means that over time there could be a change in how things are recorded, which introduces problems with our data

Question 54

Give examples of how “error of recording” could take place according to Lehrner 2002?

Answer 54

Error of recording can be due to:

          - poor technique
          - poor equipment
          - mental lapses
          - inexperience

Question 55

If we practice poor science, or there’s lots of bias, and we don’t correct for it/interpret the data with that knowledge then our findings may not be correct. What can this lead to?

Answer 55

1) A situation may be considered a welfare issue when it is not
This then leads to:
Time and money being wasted on addressing it
Time and money not used on actual welfare issues

2) A true welfare issue may not be resolved and may continue to be ignored

Question 56

If you have bias or practice poor science but still get correct findings they may not inspire confidence in the welfare issue which can lead to what?

Answer 56

Damaged reputation
Low career prospects
Low likelihood of further research in the topic
Low public confidence
Delays in resolving the welfare problem

Question 57

What are the 3 R’s?

Answer 57

Replacement
Reduction
Refinement

Question 58

What are the 3 R’s for?

Answer 58

The principals of the 3R’s were developed over 50 years ago providing a framework for performing more humane animal research

Question 59

If we practice poor science, and even if our findings are correct, nobody will believe us which means what?

Answer 59

This means the subjects would have been used for nothing

The study will also be a waste of the funder’s investment, with the money being better spent on another study

Question 60

What are the 7 things in the good scientific practice criteria?

Answer 60

1) A clear statement of hypothesis and selection of appropriate approaches to test it
2) Recognising risks of bias
3) Careful study design to minimise bias
4) Standardisation and clear description of methods
5) Discussing possible sources of bias and study limitations in publication
6) Explore alternative explanations for the findings
7) Implementation of the 3 R’s
Reduction
Refinement
Replacement

Question 61

Why is “standardisation and clear description of methods” part of the good scientific practice criteria?

Answer 61

This allows other people to:
See if potential problems were introduced
Replicate the study if necessary

Question 62

Give some examples of different observer based methods

Answer 62

Scales/rating/scoring systems
Questionnaires
Qualitative Behavioural Assessment

Question 63

What’s the 2 most important things when using observer based methods and what do they mean?

Answer 63

Precision = able to consistently get the same/very similar value
Accuracy = validity

Accuracy is the most important thing you want

Question 64

Explain how you can use scores for an observer rating?

Answer 64

You give scores for a variable you’re trying to measure using units of measurements that were defined by the researcher
The units are not standard metrics used normally like Kg or seconds

In a questionnaire the variable is also known as an item

Question 65

What can you use observer rating scores to estimate?

Answer 65

Either a:
Directly observable constructs/features
A subjective construct/feature

Question 66

What can you use observer rating scores to estimate?

Answer 66

Either a:
Directly observable constructs/features
A subjective construct/feature

Question 67

Because observer rating scores are subjective what does this mean for the results?

Answer 67

The results of observer rating scores are subjective so they can be influenced by experience and personal views

Question 68

Name examples of observer ratings that are based on 1 scale and explain them

Answer 68

Likert/numeric scales

A Likert scale will have a neutral point like this

Strongly agree _1_2_3_4_56 Strongly disagree

However Likert scales can be altered to be uni-directional like with the dog example below

These scales are categories used to judge what best describes the situation
The person making the score will select which category best represents the feature they’re trying to measure
An example is a dog ages 8yeara old will fall into the 5-10 year old category

Question 69

Give an example and explain how visual analogue scales can be used as observer ratings

Answer 69

😁 no pain 😁________________ 😫worst pain ever😫

The person making the judgement will then mark on the line where they feel it best suits the situation between the 2 ends or the category descriptors at the ends of the line
The researcher will then measure from the starting point to the point of the cross on the line
Visual analogue scales are usually 100mm long
Then the researcher can have an accurate descriptor
For example the X was at 16mm compared to 100mm

Question 70

The items or scales can be combined into what?

Answer 70

Multi-item instruments

Question 71

What can multi-item instruments be used to look at?

Answer 71

Multi-item instruments can be used to look at the combination of either:

         - several different behaviours
         - several different construct attributes

Question 72

What is the multi-item instrument called if it measures a single construct?

Answer 72

Index

Question 73

What is the multi-item instrument called if the items used are predicted to relate to eachother?

Answer 73

Scale

Question 74

What do you call a multi-item instrument if it will be used by someone other than the researcher?

Answer 74

Questionnaire

Question 75

Qualitative Behavioural Assessments are also known as what?

Answer 75

QBA’s

Question 76

Why are Qualitative Behavioural Assessments controversial approaches to studying animal welfare?

Answer 76

QBA’s are completely subjective

Question 77

What does Wemelsfelder 1997 argue?

Answer 77

1) Traditional animal behaviour methods cannot capture subjective states, such as the need flow of behaviour over time and across situations
2) We should consider subjective experiences as a perspective

Question 78

What does Wemelsfelder 1997 mean when she says “traditional animal behaviour methods cannot capture dynamic subjective states, such as the need flow of behaviour over time and across situations”?

Answer 78

Wemelsfelder 1997 argues that objective methods cause us to miss important information which humans can naturally understand by looking at dynamic changes over time in interactions with the environment
The animals dynamic changes over time and its interactions with the environment are an expression by the animal which reveals its mental state and perspective

Question 79

What does Wemelsfelder 1997 mean when she says “we should consider subjective experiences as a perspective”?

Answer 79

The behaviour and subjective experience form an expressive whole reflected in the way the “subject interacts with and pays attention to a situation”

Question 80

What does Wemelsfelder 1997 propose about Qualitative Behavioural Assessments?

Answer 80

Wemelsfelder 1997 proposed that Qualitative Behavioural Assessments assessed how the subject interacts with and pays attention to a situation, and that’s associated with welfare while incorporating elements of observer ratings

Question 81

What was the original Qualitative Behavioural Assessment called?

Answer 81

Free choice profiling

Question 82

How does Free Choice Profiling work?

Answer 82

First a large group of observers watched the animal live or on a recording
Then each observer would write down adjectives they felt best described the animals expression in the situation
The group would then take those adjectives and rewatch the video or a recording of the event they just saw and score them using visual analogue scales based on how well they believe it described the situation
They’d then use Procrustes analysis and Principal Components analysis to see if it was statistically significant or not

Question 83

Name a Qualitative Behavioural Assessment that was created in 2013

Answer 83

Fixed trait items welfare quality assessment

Question 84

Who created the Fixed Trait Items Welfare Quality assessment and why?

Answer 84

Around 2013 the Welfare Equality Project was developing welfare assessment protocols for a range of farm species
They were looking for a way to assess positive welfare since at the time there were no validated methods to do so
Qualitative Behavioural Assessments were suggested as an option but it needed to be more standardised

Question 85

How did the Welfare Equality Project 2013 standardise Qualitative Behavioural Assessments?

Answer 85

They’d usually only use a few observers
They’d ask the observers to discuss terms and reach a consensus on the meaning of those terms
Then the observers worked from a fixed list of terms to rate/score behaviour on training videos to achieve consistent responses
The observers then used the fixed terms to rate and score the animal subjects in the same way as the free choice profiling method
They’d then use the Principal Components analysis to kinda group datasets together

Question 86

Name an example of Free Choice Profiling

Answer 86

Wemelsfelder et al 2000

Question 87

Explain what happened in Wemelsfelder et al 2000’s experiment

Answer 87

Wemelsfelder et al 2000 got 2 groups of pigs, 10 pigs houses in an enriched environment and 10 pigs in an unenriched environment.
18 observers then scored individual experimentally isolated pig interactions with a familiar human in a separate test pen from the pigs housing
The 18 observers then scored the interactions using Free Choice Profiling
The data was then analysed so that the adjectives can be combined into axes/profiles that explain variation in the data
Wemelsfelder et al 2000 then took 2 profiles which explained most of the data variation and plots them on a table against each other

Then they looked at how much consensus there was between the combined axeses
In the experiment the observers significantly agreed on the pigs mental state

They then compared the profiles to what would happen if the observers randomly attributed adjectives

Wemelsfelder et al 2000 then looked at different observers judgements to see how well their judgement of the pigs matched the consensus profile
They then used the adjectives plotted on a graph to describe the axeses generated by the profiles
The researchers would look at their observers results and try to assess generally what the axeses mean

In this experiment only a few people would say the same adjective, however they assumed that different adjectives meant the same thing

Then Wemelsfelder et al 2000 would plot individual pigs on the consensus profile graph made and see how well the pigs matched the profiles dimensions
They’d then look to see if the 2 groups of pigs were actually separated along 1 or both of the axeses

Question 88

What were Wemelsfelder et al 2000’s findings?

Answer 88

Wemelsfelder et al 2000 saw no difference between the 2 pig groups in their interactions with a familiar human present

Question 89

What are the 4 limitations specific to Free Choice Profiling Qualitative Behavioural Assessments?

Answer 89

1) Free choice profiling uses lay human terms that are relevant to human emotional states, but not necessarily biologically appropriate for superficially similar states in animals
2) Free Choice Profiling doesn’t test assumptions that different observers mean the same thing with different terms

3) Free Choice Profiling tells us if the observers agree about their judgements but there is no valid test of the animals state
They’re precise about the judgement but potentially inaccurate

4) Free Choice Profiling requires prior understanding of what the specific terms mean in relation to the specific context

Question 90

What was the Procrustes analysis originally developed for and what does this mean for Qualitative Behavioural Assessments?

Answer 90

The Procrustes analysis was originally developed to look at consensus between different assessors of food quality
They were looking at features of a specific objective item
However in Qualitative Behavioural Assessments Procrustes analysis is used for a consensus on a subjective experience by another individual
As a result there is potential for bias

This is a more specific criticism of Free Choice Profiling’s validity

Question 91

What are some criticisms of Qualitative Behavioural Assessments?

Answer 91

1) The Procrustes analysis was originally designed for looking at features of a specific objective item not subjective experiences of another individual
As a result there’s potential for bias

2) many studies indicate correlation of Qualitative Behavioural Assessments with other indicators such as quantified behaviour and physiological measures
However these studies have been critized with bias through experimental design
- they may use the same people to measure the QBA and to measure the behaviour leading to an expectation bias
- some of these studies can be critised on the basis of bias or non-independence

3) the studies that use Qualitative Behavioural Assessments and show validity are usually studies where the animal was under acute challenge rather than a chronic state
4) It is also unclear if Qualitative Behavioural Assessments validity varies between species

5) some studies in the literature do still show no agreement between QBA and other indicators
However these studies have been critised for not using enough observers or using untrained observers

6) more recent work suggests that Qualitative Behavioural Assessments:
Are reliable in lab/experimental work
Not reliable in field situations

Question 92

What are the 6 advantages of observer rating methods?

Answer 92

1) They can be rapid and efficient
2) They do not require costly or delicate equipment
3) They are generally non-invasive
4) No extensive labour/time is generally needed
5) They can take advantage of familiarity with animals
6) They can allow integration of information over time so we don’t need to keep sampling

Question 93

What are the 6 limitations of observer ratings?

Answer 93

1) They may be more suitable for some species than others
2) They may be more suitable to certain traits or behaviours than other ones that are less easier to monitor or notice
3) We may not recognise the importance of some responses
4) They are subjective and so are subject to bias
5) Ratings are assigned based on observer norms and experience and so can change over time
6) Therefore we cannot assume subjective measures represent the whole picture and should be used in conjunction with other measures to ensure better validity

Question 94

What happened in Tuyttens et al’s 2014 study when they used a Qualitative Behavioural Assessments?

Answer 94

Tuyttens et al 2014 got the same students from previous tests to do a Qualitative Behavioural Assessment on videos of hens
Tuyttens et al 2014 gave the students fixed descriptors rated on visual analogue scores
The students were told that 1 video was of organic hens and 1 video was of commercial hen aviary system
However unknown to the students they were the same video again

When the students did the QBA the data showed a shift in ratings for 13 of the 19 traits looked at

This therefore shows expectation bias is also seen in the subjective scoring of Qualitative Behavioural Assessments

Question 95

What are the 6 way for reducing subjectivity and bias in observer ratings?

Answer 95

1) We need well defined, unambiguous criteria for scoring
2) Training in recognising traits or constructs and their range of expressions
3) Sufficient sampling
4) Agreement of multiple observers
5) Blinding observers to treatments to minimise expectation bias
6) we want to use validated scoring systems whenever possible