lecture 3 - action in cognition

Question 1

perception: outside-in vs. inside-out

Answer 1

• outside-in (traditional, stimulus-driven): processing information → passive sensing →. perception: sensory knowledge
• inside-out: generating information → active sensing → perception: sensorimotor knowledge
• the inside-out perspective is not just about receiving stimuli, but about acting in the world to generate meaningful experiences

Question 2

enacted perception (noë, 2007)

Answer 2

• sensorimotor knowledge plays a crucial role in perception
• to perceive one has to (implicitly) understand how sensory signals change as a function of movement (sensorimotor contingencies).
• perception is not something that happens to/within us, but it is something we do. it is not just an internal neural process, but a skillful activity that includes the body.

Question 3

empirical evidence that perception is constituted by sensorimotor knowledge (enactive theory of perception)

Answer 3

1. experiential blindness
2. no perception when images are stabilized on the retina
3. self-movement necessary for normal perception to develop
4. neuroscience studies support the inside-out view

Question 4

perception is constituted by sensorimotor knowledge: experiential blindness

Answer 4

• refers to situations where perception does not occur because of a lack of proper interaction with the environment
• for this reason, restoration of sensation (i.e., vision) alone does not guarantee perception. the brain needs the sensorimotor experience to connect what is seen with how to interact with the world.

Question 5

experiential blindness counter arguments

Answer 5

• counter argument 1: these cases of perceptual failure may result from abnormal neural connections that did not develop due to prolonged blindness, rather than because of missing sensorimotor experience.
• response 1: experiments involving goggles that distort visual input, such as making objects on the left appear on the right (or vice versa). Even though visual input is normal, this altered sensorimotor interaction leads to highly distorted perception. this further supports the idea that perception is an active, embodied process.
• counter argument 2: why is there no total experiential blindness with the goggles?
• response 2: - only sensorimotor dependencies relating to spatial interactions are affected by left-right reversing goggles. the brain can still process other elements of perception that don’t depend on spatial relationships, like color or texture.

Question 6

perception is constituted by sensorimotor knowledge: no perception when images are stabilized on the retina

Answer 6

• evidence: image stabilization on the retina leads to fading of perception.
• this highlights the importance of small eye movements (saccades) for maintaining visual perception
• this shows that perception relies on movement and environmental changes, not just static input.
• without tiny eye movements, visual perception stops, demonstrating that sensorimotor activity is crucial for perception, even at the ocular level.

Question 7

perception is constituted by sensorimotor knowledge: self-movement is necessary for normal perception to develop

Answer 7

• evidence: only kittens that could move themselves (vs. were passively moved around) developed depth and perception and ability to interact with their surroundings in a meaningful way.
• this highlights that self-movement is necessary for testing and learning sensorimotor dependencies
• action (and sensorimotor knowledge) thus seems a constitutive feature of perception

Question 8

perception is constituted by sensorimotor knowledge: neuroscience studies support inside-out view

Answer 8

• brain areas for motor control and action planning are also involved in perception
• suggests the brain actively anticipates and interacts with sensory information, rather than passively processing it

Question 9

arguments against enacted perception

Answer 9

1. not all visual processes rely on (sensorimotor) action
2. paralyzed people are not blind

Question 10

objection: ‘not all visual processes rely on (sensorimotor) action’ + response

Answer 10

OBJECTION

there are two distinct visual pathways in the brain

1. ventral pathway (what pathway): identifying and recognizing objects
2. dorsal pathway (where/how pathway): guiding movements related to objects

• visual agnosia: damage to the what pathway leads to an inability to recognize objects, but can still act on them (visuomotor skill still intact)
• damage to the were pathway leads to intact vision, but impaired visuomotor skill
• This separation of functions shows that object recognition and motor action are processed differently in the brain, indicating that not all visual processing requires sensorimotor engagement.

RESPONSE

• misunderstanding of enactive view: The enactive view asserts that perception is built upon sensorimotor knowledge, not just action.
  even in cases where people cannot use visual information to guide action (e.g., visual agnosia), they still have sensorimotor understanding of how their movements (e.g., eye) influence sensory input.
• all perception depends on movement-based knowledge: whether the result of dorsal or ventral stream activity, all forms of perception are shaped by the knowledge we’ve gained through movement (i.e., deployment of sensorimotor skills). This means perception always involves some form of sensorimotor understanding.
• ventral stream and visual object recognition: though the ventral stream is classically associated with object recognition and visual processing of “what” things are, it might also be involved in processing locomotive affordances—the possibilities for movement offered by an environment (PPA, OPA).

Question 11

objection: ‘paralyzed people are not blind’ + response

Answer 11

OBJECTION

if enacted perception (the idea that perception depends on movement) were true, then paralyzed people should be unable to perceive properly. However, paralyzed people are not blind and can still perceive the world, even without being able to move their bodies.

RESPONSE

• paralyzed people can still move their eyes and heads. This means they can still engage in movement to some extent, and this limited movement helps them perceive their surroundings.
• even minimal movement, like moving one’s eyes or head, can provide enough sensorimotor information to help maintain perception.
• Quadriplegics are continuously engaged in the task of orienting themselves in relation to the world around them and to gravity.

Question 12

classical view: cognitivism

Answer 12

• cognitive systems are sandwiched between environmental inputs and outputs
• this means that cognition (e.g., thinking etc) is believed to happen inside an individual’s mind or brain, separate from the external world.
• interaction with the environment is limited to taking inputs and producing outputs.
• this emphasizes the internal nature of cognitive processes
• i.e., cognition = internal computations on mental representations

Question 13

traditional view of action (according to barsalou)

Answer 13

• action is little more than making responses.
• this suggests that action plays a minor role in cognitive theories and experiments
• cognition is primarily about internal processes that response to external stimuli, and action is just the outcome of this process
• the biggest concerns about action are things like counterbalancing handedness, or preventing movements during neuroimaging

Question 14

pragmatic turn: 4E movement

Answer 14

• represents a shift from the traditional view of cognition being purely internal to one that sees cognition as being shaped by dynamic interactions between the brain, body, and environment.
• meaning that an action oriented paradigm emerged in cognitive science

1. embodied
2. embedded
3. enacted
4. extended

Question 15

4E: embodied cognition

Answer 15

• The brain doesn’t work in isolation includes the body’s structures and processing in shaping how we think and perceive
• For example, our physical movements, senses, and bodily experiences influence our thoughts and how we interact with our surroundings.

Question 16

4E: embedded cognition

Answer 16

• cognition is not isolated from the environment, but functions only when embedded in a relevant external context
• The environment provides important context for how we think and act. We understand things better when we account for the organism’s interaction with its surroundings.

Question 17

4E: enacted cognition

Answer 17

• involving not only neural processes, but also what an organism does
• actions help shape our cognitive processes, meaning cognition is partly constructed through the things we do, not just the things we think
• highlights the importance of interaction with the world

Question 18

4E: extended cognition

Answer 18

• cognition can be extended beyond the brain and body, into the environment
• e.g., smartphones, notebooks, or even social interactions
• suggests that thinking and problem-solving can involve the use of external elements, and these are intergral parts of how cognition functions

Question 19

enactivism

Answer 19

• the mind is not made up of representations in the head, but the mind is relational
• thinking is not confined to the brain. although it is crucial for cognition, cognition emerges from the interaction between the brain, the body, and environment
• Cognitive processes don’t just find or copy things that already exist in the world. Instead, they help us create and shape how we see and understand the world based on what is useful for our actions and interactions with our surroundings.
• opposes the classical view of cognition that centers around the manipulation of internal representations. instead, it proposes that cognition emerges from the active engagement of an organism with its environment, emphasizing action, embodiment, and situatedness.

Question 20

cognitivism → enactive view

Answer 20

• cognition is understood as computations over mental representations
• the subject of cognition is a detached observer with a bird’s eye view of the world
• intentionality is explained by the representational nature of mental states
• the architecture of cognitive systems is conceived as being highly modular
• processing in subsystems is assumed to be largely context-invariant
• computations are considered neutral to the body or environment (functionalism)
• models of cognition take into account only the inner states of a cognitive system, and ignores how the body or environment might affect thinking (individualism)

Question 21

cognitivism → enactive view

Answer 21

• cognition is understood as capacity of generating structure by action
• the cognitive agent is immersed in his/her task domain
• system states acquire meaning by virtue of their role in the context of action
• the functioning of cognitive systems is thought to be inseparable from embodiment
• a holistic view of the architecture of cognitive systems prevails, which emphasizes the dynamic nature and context-sensitivity of processing
• models of cognition take into account the embedded and ‘extended’ nature of cognitive systems

Question 22

enactive view: internal representations

Answer 22

• internal representations come about through interaction with the environment, so by definition, are dynamic, context dependent, and include the body
• therefore, as a consequence of the enactive view, there are no fixed, context-invariant internal representations/ an object concept is not an internal description of this object
• to know an object is to master sets of sensorimotor skills and possible actions that can be chosen to explore or utilize the object
• therefore, object concepts and representations are shaped by the actions we can perform with them (affordances) and the context in which they are encountered

Question 23

dynamic directives (engel et al., 2013)

Answer 23

• dispositions for action, embodied in dynamic patterns of activity
• i.e., they are action patterns that guide how we interact with the world. These action patterns are not fixed ideas but flexible and changing based on our body and the environment.
• encoded not just in motor regions, but also in sensory and memory regions, showing that our actions are shaped by our whole body and the situation we are in.
• not just internal states in the brain, but also include bodily dynamics (arising from biophysical properties of the skeletomuscular system) and part of the environmental niche
• moves away from representations as mental pictures and replaces it with a term that does not carry the cognitive burden
• coincides with the finding of locomotive affordances in the ventral stream

Question 24

improbable object

Answer 24

these show how the absence/distortion of affordances related to an object lead to confusion in understanding of how to use that object

Question 25

engel: ‘can sensorimotor contingencies be exploited to acquire abstract cognitive concepts’

Answer 25

• this is a question about the definition of enacted cognition
• if sensorimotor contingencies could be applied to abstract concepts, it would suggest that even abstract reasoning is somehow grounded in embodied experience, and that our understanding of abstract concepts might be linked to methaphorical or indirect forms of sensorimotor experience

Question 26

pragmatic implications of the enactive view (methodological shift)

Answer 26

• if neural states are individuated through their role in action generation, primary focus of experimentation should be on studying the relation of neural activity patterns to action contexts, rather than investigating their dependence on external stimuli
• avoid studying passive subjects, but use paradigms that require novel technology to use paradigms that involve active exploration

Question 27

premotor theory of attention

Answer 27

• the frontal eye fields in the premotor cortex are active during selective attention, as well as during eye movements. this emphasizes how attention and action planning are linked in the brain
• selection of sensory information (what we attend to) is shaped by constraints that arise from current action planning and execution
• suggests that the way we select what to focus on is influenced by the movements we are planning to make. the brain uses movement-related areas to decide where to focus attention.
• attention is not just about what we “see” but also how it is influenced by our body’s readiness to act

EVIDENCE

1. covert attention relies on the same premotor regions as overt attention, which indicates a shared neural mechanism.

• but: The FEF contains motor neurons, visuomotor neurons, and visual neurons. This mix shows that the FEF plays a role in moving the eyes, processing visual information and focusing attention.

2. movement preparation can lead to attentional shifts.

• but: attention can shift to locations not reachable with the eyes, which suggests that attention is not always tied to the physical constraints of the body

Question 28

Barsalou: in-between view

Answer 28

• says that the idea of static, invariant representations is a caricature. the representations neuroscience generally refer to a systematic relationship between features of the natural world and the activity of neurons in the brain, but this doesnt mean there is only one invariant representation
• barsalou offers an in-between view;

1. there is no permanent static representation of hammers in long-term memory, built from amodal symbols, that is loaded into WM identically across different occasions
2. the representation that does reside in long-term memory results from superimposed effects of associative learning distributed across relevant sensorimotor systems, with the resultant network changing constantly after every learning episode (and overlapping considerably with networks for other categories)
3. when this distributed network is accessed, it produces one of infinitely many hammer representations dynamically
4. these representations serve temporary representational functions by providing useful inferences in specific situations

Question 29

evidence for barsalou’s in-between view

Answer 29

studies show that WM representations are distributed and include sensory and premotor regions

Question 30

barsalou: cognition cannot be reduced to modal (sensory/motor) systems. it mediates between sensation and action

Answer 30

1. cognition can represent non-present states
2. cognition can create conceptualization through language

Question 31

barsalou: cognition can represent non-present states

Answer 31

• humans can represent nonpresent states internally (in WM, imagery, what if scenarios, hypotheticals)
• this doesnt imply static representations in long-term memory, but dynamic multimodal representations that are temporarily constructed in WM to represent nonpresent states
• doing this is central to expanding the action reprertoire of humans

Question 32

barsalou: cognition can create conceptualization through language

Answer 32

• cognition mediating between sensation and action allows humans to interpret the world conceptually in ways that support novel and powerful action
• allows goal directed categorization to conceptualize nonpresent objects and events
• example: reappraising situations and conceptualize foods to make informed decisions and change actions

Question 33

implication of barsalou’s view

Answer 33

• without a mediating process, actions directly reflect triggering stimuli, encoded by evolution as hard-wired responses
• cognition it mediates between stimuli and responses and therefore exists to support adaptive action, rather than deterministic action
• this way, it becomes possible to change responses to the same stimulus in ways that optimze outcomes for the system in its environment

Question 34

barsalou: to understand cognitive processes

Answer 34

it is important to understand not only its constant entwinement with action, but also how action contributed to creating a cognitive process

• while mediating processes cannot be reduced to action, they could not develop without action, and action is deeply implicated in the forms they take, the expression of these mediating processes occurs through action

Question 35

free energy principle: cognitivism vs. enactivism on richness of representations

Answer 35

1. cognitivist/helmholzian account: the predictive mind consists of representations so rich and reconstructive that they enable the organism to ‘throw away the world’ (internal mirror of nature)
2. radical PP: the brain is not much more than a lazy prediction machine, allowing us to represent just enough to make the most adaptive actions and is thus better fit for survival. generative models are geared to promote fast and successful engagements with the outside world and sensing delivers an action-based grip on, rather than a mirror-like image of, the world.