Joint source channel coding

Question 1

What is the difference between source coding and channel coding?

Answer 1

Source:
Convert source symbols into sequences of equally probable bits.
R ≥ H(X)
Channel:
Transfer messages of equiprobable bits over noisy channel by adding redundancy.
R ≤ C

Question 2

Layered architecture

Answer 2

Leads to scalable systems. Not always optimal or the simplest,
but scales well. The role of source coding (high layer) is to pass uniformly random bits to the channel coding (low layer).

Question 3

Shannon’s Source–Channel Separation Theorem

Answer 3

Given a discrete memoryless source, (U, p(u)), discrete memoryless channel, (X , p(y|x), Y), and distortion measure, d(u, ˆu):

• if r * R(D) < C , then (r, D) is achievable, and
• if (r, D) is achievable, then r * R(D) ≤ C

We use separate lossy source coding and channel coding

Source coding: For any ε > 0, there exists a sequence of lossy source code with rate R(D) + δ(ε) that achieve average distortion ≤ D
We treat the index for each code in the sequence as a message to be sent over the channel
Channel coding: The sequence of source indices can be reliably transmitted over the channel provided r(R(D) + δ(ε)) ≤ C − δ′(ε)
The source decoder finds the reconstruction sequence corresponding to the received index. If the channel decoder makes an error, the distortion is ≤ dmax. Because the probability of error approaches 0 with n, the overall average distortion is ≤ D

Question 4

Give an example of uncoded transmission

Answer 4

Consider sending a Bern(1/2) source over a BSC(p) at rate r = 1 with Hamming distortion ≤ 𝐷.
If p ≤ D, then send uncoded.