Week 7 Flashcards
Applications of AFA for detection of physical pain
1) subjective
2) patient self-report with limitations:
- idiosyncratic
- susceptible to suggestion
- deception
- unsatisfactory with people incapable of articulating their feeling
3) Using behavioural measures ⇒ facial indicators of pain: brow lowering (AU4), orbital tightening (AU6 & AU 7), nose wrinkling & lip raise (AU9 & AU10), eye closure (AU43)
How could automated face analysis aid in diagnosing and treatment of depression
and psychological distress?
1) facial expression & other nonverbal communication
⇒ indicators for disorder severity & response to treatment
2) depressed individuals:
look less at conversation partners, gesture less
smile less, more smile suppressor movements,
less facial animation
3) replicated using AFA ⇒ useful for screening efforts in mental health
4) screening
what does polygraph do?
monitors uncontrolled changes in heart rate & electro-dermal
response
What are the limitations of polygraph?
1) continuously connected to subject’s body
2) requires accurate calibration to establish baseline measurements
3) an overt system, i.e., subject aware
4) requires a trained operator, who controls the likelihood of human error/length of interview, get tired
How can thermal imaging address limitations of polygraph?
1) Computes mean temperature of the 10% hottest pixels from within the periorbital region of interest (ROI)
- mean temperature on the vasculature in inner corners of the eyes
2) consists of:
- Low varying component indicative of long term trend of blood flow
levels
- Mid frequency component, which is associated with temporary
disturbances in blood flow caused by stress in specific Question &
Answer (Q & A) sessions
- High frequency component caused by tracker instability & systemic
noise
how does stress and deceptive behaviour manifest during interrogation?
in peripheral senses through various
physiological signatures, e.g., perspiration, pulse, &
breathing rate - basis of polygraph
How does thermal imaging of facial physiology work?
1) Correlation of increased blood perfusion in the orbital
muscles & stress levels
2) Periorbital perfusion can be quantified via the processing of thermal video based on:
- skin temperature is modulated by superficial blood flow
- heat convected by blood flow in ophthalmic arteriovenous complex is responsible for elevated temperature w.r.t. the rest of the periorbital region.
- supply of additional blood to eye muscle is realised through this complex
- ⇒ monitor the conduit in the eye corners to detect stress
How can the video-based detection of head motion, facial expression and body
motion be used to recognise deception?
1) tracks movements of subjects’ hands & head relative to their body
2) analyses some facial expressions & their 3D head pose ⇒ motion profile
3) All motion is histogrammed into 5 bins, with each bin having an exponentially increasing size
i. e., bin 1 covers a very small range; bin 5 covers the largest range
How can interpersonal coordination between a mother and an infant be monitored?
1) pattern of association for head motion and AUs between mothers
& infants - non stationary (i.e., mean & variance of the underlying
process are not constant)
2) non-stationarity in head pose coordination of distressed intimate
adults
3) head amplitude and velocity for pitch (nod) & yaw (turn) - strongly
correlated between them, with alternating periods of instability (low correlation) followed by brief stability in which one or the other
partner led the other
Describe windowed cross correlation between mother and infant head-pitch amplitude
1) Area above midline (Lag > 0) - relative magnitude of correlations for which the
mother’s head amplitude predicts her infant’s
2) Area below midline (Lag < 0) - the converse
3) Midline (Lag = 0) - both partners change their head amplitudes at the same time
4) Positive correlations (red) - head amplitudes of both partners change in the
same way (i.e., increasing together or decreasing together)
5) Negative correlation (blue) - head amplitudes of both partners change in the
opposite way (e.g., head amplitude of one partner increases as that of the other
partner decreases)
6) Direction of the correlations changes dynamically over time
How can automated face analysis be used in marketing?
Using web-cam technology to record thousands of viewers in dozens of countries, and to process their facial
expression to infer liking or disliking of commercials &
products
How can automated face analysis be used in instructional technology?
1) Interest, confusion, rapport, frustration, and other emotion & cognitive-emotional states - process variables in classroom & in tutoring
2) Ability to distinguish between closely related facial actions that signal student’s cognitive-emotional states
How can automated face analysis be used in computational behavioural science?
1) In conversation, expectations about another person’s identity are closely involved with his or her actions
2) Over telephone, inferences are made from the sound of the voice about the other person’s gender, age & background
3) An individual has a characteristic & unified appearance,
head motions, facial expressions & vocal inflection
How can automated face analysis be used in media arts?
1) Widely used in the entertainment industry
2) Movies, e.g., Avatar & Hobbit
3) Gaming, e.g., Sony’s Everquest II
how does body better communicate some affective expressions
body posture communicates cause of a threat & the following action, whereas face communicates only the threat
Discuss the advantages and disadvantages optical motion capture
1) accurate numeric representation of body in 3D space
2) anonymous data
3) not portable
4) marker occlusion
5) high cost
Electromechanical motion capture system
1) uses potentiometers on plastic exoskeleton that the subject wears
2) exoskeleton tracks human joints &
angles between body segments to
reconstruct the 3D body
Electromagnetic motion capture system
1) electromagnetic sensors placed on body
2) measures orientation & position of sensors relative to electromagnetic
field generated by a transmitter to reconstruct the 3D body
3) no problems with occlusion, but EM interference
Benefits of electromechanical and electromagnetic systems
1) accurate numeric representation of
body in 3D space
2) anonymous data ⇒ privacy
3) portable ⇒ can be used in almost any setting, indoors or outdoors
4) much cheaper than optical motion capture system
Markerless based-vision systems
1) use video or web cameras to record movement
2) no mobility issues
environmental conditions pose challenges:
variations in lighting, skin colour & clothing body part occlusion or touching
3) non-intrusive
Describe the operation principles of the Microsoft Kinect sensor, and how it could be used to recognise body expressions.
1) a depth sensor & a colour camera
2) depth sensor: IR projector & IR camera (a monochrome CMOS sensor):
- based on structured light principle
- IR projector - IR laser that passes through a diffraction grating & turns into a set of IR dots
- known relative geometry between IR projector, IR camera & projected IR dot pattern
- compute depth map using 3D triangulation of corresponding dots in
image & projector pattern
What are the cross-cultural similarities and differences in perceiving affect through
the human body?
1) differences due to social status being more important to Japanese (than American)
2) similarities found: sad/depressed postures (Japanese, Sri Lankan, American)
3) differences in assigning intensity ratings to emotions:
Japanese assigned higher intensity (Japanese, Sri Lankan, American)
4) dimensions required for seated posture:
- Japanese - 3 (arousal, valence, & dominance)
- British - 2 (arousal, valence)
Discuss the usefulness of body features for emotion recognition
1) motion signals - sufficient for recognition
2) recognition accuracy is impaired when form information is
disrupted, e.g., by inverting or reversing the motion
3) posture cues aids in discriminating between emotion with similar dynamic cues or movement activation
4) dynamic information - complementary & partially redundant
to form
5) possible to classify numerous affective behaviour using only
upper-body features
5) meaningful groups of emotions could be clustered in 1/4 quadrants of valence/arousal plane (Glowinski 2011)
State 4 groups of body expressions in Dael, Mortillaror & Scherer (2012) coding system that are useful in distinguishing between emotions perceived via body action and posture.
1) Head orientation
2) Head posture, e.g., lateral head turn towards left position
3) Trunk orientation, i.e., facing & averted
4) Trunk posture, e.g., trunk lean towards a forward position
5) Whole body posture, e.g., whole body moves or leans towards a backward position
6) Arms posture, e.g., left arm at side
7) Gaze, i.e., toward, upward, downward, averted sideways, eyes
closed
8) Head action, e.g., upward head tilt
9) Trunk action, e.g., spine bending
10) Arm action, e.g., left arm action away from the body
11) Other, i.e., touch, knee bend & leg movement
12) Action function, e.g., beat (i.e., repetitive action that accentuates
points in time, illustrating structural or rhythmic aspects of co-occurring speech)
