Lecture 1: Prefix Codes

Question 1

What is meant by a uniquely decodable code?

Answer 1

if C+(x) != C+(y) where x != y. Essentially, if two input strings (x and y) were to encode to the same codeword,
you can’t tell whether it was x or y as the input.

Question 2

What is a instantaneously decodable code? How do we generate them?

Answer 2

When it is possible to instantly decode a codeword as soon as we receive it, without having to look ahead for future codewords. Generate them using prefix codes.

Question 3

Can a code be uniquely decodable without being a prefix (instantaneous) code?

Answer 3

Yes, say C = {1, 101}. Although 1 is a prefix of 101, only 101 contains a 0, so can distinguish the codewords with that.

Question 4

When using a prefix code as a binary tree, what can you not do?

Answer 4

Once a node on the tree is selected as a codeword, no further child nodes can be also be selected.

Question 5

What is an optimal code?

Answer 5

Using all possible codewords for a prefix code (using all the budget of a budget grid).

Question 6

Explain the Kraft Equality

Answer 6

For lossless compression, codewords cannot be arbitaraliy short, shortening one must cause another to lengthen. A code is ‘complete’ if the equality holds (=1).

(Sum of 2^-len(k) for all codewords <= 1).

Question 7

How does the Kraft Equality apply to budgets/binary tree?

Answer 7

All node endings are used, all the budget is used. Complete code.

Question 8

Discuss the entropy bound on data compression.

Answer 8

The code produced must be greater or equal length to the Shannon entropy! (Limit of data compression)