10_Information Theory Flashcards

1
Q

What is the purpose of redundancy in data?

A

Redundancy in data helps detect and correct errors by reducing entropy and increasing predictability.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is channel capacity in information theory?

A

Channel capacity is the maximum rate of data transmission over a channel without errors, measured in bits per second.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What does the Noiseless Coding Theorem describe?

A

It describes the limit to which data can be compressed without losing any information, defining the minimum size of encoded data.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

How does the Shannon-Hartley theorem apply to communications?

A

It connects a communication channel’s capacity to its bandwidth and signal-to-noise ratio, calculating the maximum data rate for noisy channels.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is mutual information?

A

Mutual information measures how much knowing one variable reduces uncertainty about another, quantifying information shared between variables.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Define source coding.

A

Source coding is compressing information into fewer bits than the original representation to make data storage and transmission more efficient.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

How does information theory impact network theory?

A

It optimizes data flow and routing in networks by applying principles of efficient data transmission and error handling.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are error-correcting codes and their significance?

A

Error-correcting codes add redundancy for detecting and correcting transmission errors, crucial for keeping data integrity.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is rate-distortion theory?

A

Rate-distortion theory studies the trade-off between the rate of data transmission and the quality distortion, important for understanding limits of compression.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

How is information theory applied in cryptography?

A

Information theory helps design secure cryptographic systems by ensuring keys and ciphers are unpredictable and robust against decoding.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly