lecture 5 - WM in action

Question 1

what does the cognitive brain do (psychology & ecology approach)

Answer 1

1. psychology approach: perception → cognition → action
   - linear process
2. ecology approach: action specification + selection
   - instead on a linear process, action is prioritized based on immediate needs
   - attention, WM, and other cognitive processes are seen as supporting tools that help achieve action, not as standalone steps that must be completed before acting

Question 2

implications of the ecology approach

Answer 2

1. action is not the end-point, but the starting point: we recruit cognition because of action goals and action planning is likely to operate alongside cognitive processes.
2. cognition is likely ‘rooted’ in brain circuitry that initially evolved to guide action: action circuitry may control things that appear purely ‘cognitive’. i.e., action existed first, and cognition evolved later to guide adaptive behavior
3. cognitive functions guide behavior together, and should therefore be studied together: i.e., rather than in isolation
4. it is sensible to study cognition in its ‘natural habitat’ of behaving humans
5. when isolating cognitive functions from action, you may end up studying things that do not generalize well to real-life cognition and forget to ask relevant questions: i.e., action and cognition are deeply intertwined. therefore, we consider behavior as a whole, not individual cognitive processes isolated from the context of action

Question 3

purpose if the mind

Answer 3

to serve behavior

Question 4

selective attention

Answer 4

• WM runs parallel to the sensorimotor arc, serving to inform potential and sequential upcoming behavior.
• selective attention in WM serves to prioritize internal contents (representations) (inside-out) and external content (outside-in) for behavior

Question 5

purpose of WM

Answer 5

• in everyday life, selecting informational content and pragmatic guiding of action are intrinsically connected. this linking may be the defining purpose of working memory
• i.e., working memory and selective attention link mind to behavior

Question 6

retrocue task

Answer 6

participants retain and encode information of two visual objects before a color retrocue indicates which object from memory will (most likely) become relevant for guiding ensuing behavior

Question 7

anticipating WM use in time experiment

Answer 7

• in this task, participants view two stimuli simultaneously (yellow and blue), then after a time delay, they are asked to recall the orientation of the stimuli in a sequential order. the yellow stimulus is probed first 80% of the time, while the blue one is probed 20% of the time.
• the results show that when the blue stimulus is unexpectedly probed first, participants take longer to recall its orientation
• this indicates that WM adjusts its prioritization, which highlights the flexible, nature of selective attention inside WM for guiding action
• so, in contrast to external attention, selectively prioritizing content A in WM does not always imply that content B will be forgotten/impoverished. B can be retained in a temporarily unattended state and still be reprioritized later.

Question 8

selective attention inside working memory (internal selective memory) is

Answer 8

1. functional: visual memory + action plan
2. flexible: prioritization and reprioritization of content in working memory
3. forward looking (future focused): before retrocue studies it was believed that WM was of a fixed nature

Question 9

traditional view of visual WM vs current view

Answer 9

• traditional: WM is just for holding visual information because otherwise this information would disappear (i.e., has a fixed nature)
• current: WM is for guiding action, facilitated through action, and as a result of moving around

Question 10

current view of visual WM

Answer 10

• that it is for guiding specific actions (flexible and future focused)
• for this reason, we should investigate visual representations - and their selection - in contexts where they are associated with specific actions

Question 11

real world laboratory experiment

Answer 11

• participants see a line in a real-world simulation, hold it in WM, and recall the location of the line verbally (visual memory), or the orientation with either hand (visual + motor memory)
• if the primary/only information maintained during a visual WM task is the sensory (visual) content, participants would first have to select the relevant object when prompted to response, and only then formulate the appropriate action plan
• however, markers of visual-spatial selection and motor-plan selection occurred concurrently.
• conclusion: WM is not just about sensory, content-related representations. pragmatic representations are also encoded and maintained, ready for guiding performance.
• visual working memory and motor preparation are distinct but interconnected processes, each activating different regions in the brain
• highlights the functional nature of internal selective attention

Question 12

when do action plans form during WM

Answer 12

• output planning is done at the input stage
• output planning predicts performance 6 seconds later, meaning that action planning is already underway during the encoding phase, before the go-cue is given

Question 13

action plans are

Answer 13

1. tuned to relevant moments
2. brought into working memory immediately
3. held available for selection simultaneously

Question 14

why could action-ready memories make a difference

Answer 14

1. precision of action
2. speed of action: faster memory guided behavior by having action plans ready
3. parallel plans for multiple potential scenarios: multiple sources of selective attention can work in tandem to affect performance simultaneously

Question 15

the impact of memory load on action initiation

Answer 15

• memory set-size affects preparedness, not accessibility
• with more contents in the mind, actions are initiated slower, not due to difficulties in accessing visual content, but due to reduced motor readiness.
• visual selection over time is similar, regardless of memory load. this suggests that accessing visual content for memory is not significantly delayed, even with a higher load
• however, with a higher memory load, there is less motor preparation over time, meaning the brain is less ready to act when holding more information in WM

Question 16

oculomotor system

Answer 16

• frontal eye fields (FEF) and superior colliculi (SC)
• these are part of the action circuitry that controls eye movements

Question 17

oculomotor system control of external covert attention

Answer 17

• the oculomotor system provides a major source of top-down control over the spatial allocation of covert attention

1. FEF is a key node in the dorsal attention network, which is responsible for goal-directed attention. the FEFs role in this network is to help allocate attention spatially, even when there is no overt eye movement.
2. eye-movements automatically shift attention to the location the eyes are about to move toward. this means that eye movements affect where attention is, and drive attention shifts.
3. stimulation of FEF and SC facilitates visual processing: visual activity increases after microstimulation, suggesting a strong link between the oculomotor system and improved visual perception
4. SC inactivation impairs selective attention: specifically, it leads to impairment of the ability to detect a moving object

Question 18

external perception

Answer 18

• control of the overt gaze and covert attention are naturally linked
• inputs at covertly attended locations can inform where to look next
• our ability to focus attention externally, even without eye movements, is deeply connected to the same brain systems that guid eye movements

Question 19

oculomotor system control of internal focus of attention

Answer 19

1. in an experiment where participants are to report the rotation of two stimuli, their gaze tends to shift toward the remembered locations even though they are not asked about stimulus locations. their eye movements reflect where the remembered items are located, even though they aren’t explicitly looking at them.
2. microsaccades: the rate of microsaccades increases after a cue is presented, and these microsaccades move toward the remembered location shortly after the cue. These microsaccades serve as indicators of attentional focus, even when processing internal information (locations of objects maintained in WM).

Question 20

practical implications of bias in gaze and microsaccades

Answer 20

1. reveals an oculomotor contribution to internal selective attention
2. provides a means to track internal attention

Question 21

studying how we prioritize and switch between items held in WM

Answer 21

• done with a look ahead fixations experiment
• task is designed to observe how the brain shifts focus within working memory, prioritizing items in the correct order
• a cue is provided to indicate which of the two items should be prioritized for retrieval first. the participant then reports that item, and then reports the second item
• spatial gaze bias: gaze shifts towards the location of the item that is currently prioritized (i.e., looking ahead)

Question 22

implication of the spatial gaze bias (looking ahead)

Answer 22

1. shift in gaze bias (from first to second stimulus) represents the mental switch within WM from the first reported item to the second
2. demonstrates how gaze bias is linked to internal prioritization in WM
3. reflects how attention and WM adapt to task demands in real time

Question 23

looking ahead: IKEA example

Answer 23

• When assembling something complex, like a tent, people tend to plan their next steps visually and mentally. This involves looking ahead to the next step in the process as soon as the brain can “afford” to do so.
• This behavior mirrors how the brain dynamically switches priorities in working memory tasks, exploring upcoming steps before they are needed
• i.e., WM brings relevant information from the past forward in order to prepare for the future

Question 24

WM as a consequence of action

Answer 24

• WM exists because we move around, not because visual information disappears
• i.e., due to our movements, we require constant updating of our memory to account for shifting spatial frames.

1. VR experiments show participants preserve multiple spatial frames of items
2. brain holds on to information about past events and makes future predictions at the same time

Question 25

multiple spatial frames

Answer 25

• when we move, items in our WM exist in multiple spatial frames. they are no longer fixed in a single location
• as we move, the brain has to keep track of these shifting frames of reference, which creates a dynamic, action-driven WM system
• now memoranda may exist in multiple spatial frames

Question 26

immersive WM in VR experiment

Answer 26

• allows researchers to study how working memory operates in a 3D, real-world-like environment, where the spatial aspect can be manipulated and measured more precisely
• gaze moves toward the remembered left or right stimulus location after the cue.
• multiple spatial frames for selection: participants look at remembered items in different spatial frames of reference to remember the position of items in their environment (i.e., preserving a spatial mapping of the item in its environment)

Question 27

experiment: memorizing moving objects

Answer 27

• shows the process of jointly looking at both the past and the future in WM
• the brain simultaneously holds on to information about past events and future predictions
• this helps e.g., with holding moving objects in mind

Question 28

take home messages

Answer 28

1. WM looks forward, which enables informed upcoming behavior
2. selective attention in memory prioritizes contents for behavior
3. visual working memory does not precede action, it incorporates action
4. cognitive functions work together to support adaptive behavior
5. temporal profiles contain vital clues for unravelling the workings of the mind

Question 29

visual WM for action

Answer 29

• visual WM helps guiding appropriate action

1. anticipates WM use in time by holding objects and prioritizing/reprioritizing as needed
2. visuospatial processing and motor plan selection happen at the same time

Question 30

visual WM through action

Answer 30

• action facilitates working memory (attention) through the oculomotor system

1. the oculomotor system links control of covert gaze and external attention
2. the oculomotor system controls internal attention by showing a gaze bias and looking ahead toward the attended location