Entropy Flashcards
1
Q
Prove that the entropy of a discrete random variable X with alphabet cardinality M must satisfy the constraints 0 ≤ H(X) ≤ log M
A
2
Q
Definitions of code types
A
3
Q
Theorem (Kraft-McMillan)
A
4
Q
Shannon’s First Theorem
A
5
Q
Shannon coding + 1 bit
A
5
Q
Prove that Huffman codes are optimal instantaneous binary codes
A
6
Q
If code C’ is optimal, then code C is optimal
A
7
Q
Run-length coding
A
8
Q
Asymptotic Equipartition Property
A
9
Q
Typical set
A
10
Q
Arithmetic coding
A
11
Q
Scalar Quantization
A
12
Q
Uniform Quantizer
A
13
Q
Optimal Uniform Quantizer
A
14
Q
Lloyd - Max Quantizer
A
