Information theory

Question 1

what is the main description of information theory?

Answer 1

• about noise (anything causing ambiguity/ interference in communication) relative to communication
• information is what we communicate

Question 2

what kind of theory is information theory?

Answer 2

-normative theory, interested in the efficiency of the process

Question 3

what is information theory sensitive to? (2)

Answer 3

1. time (sequence of events, time series, stochastic processes, probability in time)
   - behaviour changes in time and space (dynamic and stochastic)
2. context (noise, other stimuli)
   - Ex. coyotes don’t howl on windy nights

Question 4

what is the focus of information theory?

Answer 4

• focus=transmission

- also interested in reception

Question 5

what is information in information theory? (7)

Answer 5

1. choice/ difference –>information offers a choice in stimuli produced
2. order–> communication often not orderly
3. pattern
4. entropy (disorder, mostly due to noise)
5. redundancy (ex. English better in noisy channel as less complexity)
6. uncertainty/ probability –> flexibility, adding degrees of freedom allows change
7. omission–> can define information by its absence

Question 6

Terrence Deacon

Answer 6

• definition of information by omission
• absence of details is as telling as those that are present
• info becomes evident when we don’t have it

Question 7

omission in regards to information

Answer 7

• info becomes evident when we don’t have it
• amygdala picks up on missing information at the unconscious level (likely have theory of mind) –> even when aware may not be able to tell what is going on
• ability to pick up the difference

Question 8

difference between subcortical and cortical parts of brain in animals

Answer 8

• subcortical parts of brain are similar between animals, yet cortical are very different (social behaviors may be regulated by basic subcortical processes)
• evidence of divergent evolution
• evolution of birds and mammal brains, foundation start in reptiles

Question 9

what did Walter Freeman suggest about information?

Answer 9

• information is good, but sometimes there is too much

- what is important is the meaning/ relevance of the information

Question 10

what kind of system is used within information theory? describe what it is?

Answer 10

• A binary system (0s and 1s)
• signifies the presence or absence of either a 1 or 0, which represents a corresponding symbol
• gets rid of irrelevant info and decide what is relevant
• each digit is a bit (binary digits)–> analogous to a coin toss

Question 11

entropy

Answer 11

=amount of information you have

-look at symbols used in message (how much you have to process) and symbols available in language/code (eg. size of alphabet)

Question 12

Information for Shannon

Answer 12

• refers to entropy; reducing the probabilities
• give unexpected information
• average logarithm of the improbability of the message, measure of unexpectedness/ surprise

Question 13

how does redundancy affect information?

Answer 13

• redundancy is the antidote to confusion (if confused then we repeat)
• more redundancy= less efficiency but better error correction
• more a msg is regular/has order= more predictable= more redundant
• more redundant=contains less info (reduces confusion)

Question 14

how do uncertainty and information relate?

Answer 14

• more information= less uncertainty

- uncertainty increases with number of possible alternatives