Information and Telecommunication Systems

Question 1

What is information?

Answer 1

• Information is a mathematical, descriptive characteristic of the message that can be defined as the “average amount of surprise” involved in receiving a long stream of messages from a source with particular, known characteristics.
• “Information is what makes us aware of something we did not already know – it is the difference between our knowledge at one time instant and the next.”

Question 2

The connection between information and communication?

Answer 2

There is a fundamental connection between information and communication: a fact only becomes useful as information when it is communicated.

Question 3

What is a signal?

Answer 3

A “signal” is the actual entity (acoustic, electrical, mechanical, optical, etc.) that is transmitted from sender to receiver,

Question 4

Define a message?

Answer 4

• In formal language, a message can be defined as the particular set of values being conveyed.
• A “message” is the knowledge transmitted

Question 5

The difference between message and information?

Answer 5

Message and information are very similar but not identical: information is the newness or surprise within a message, e.g. for all the birds within some vicinity to hear that the mating season has arrived is no surprise, but that a particularly attractive bird is available to mate, maybe!

Question 6

Example signal, message and infomation present in a FAX message?

Answer 6

The image is transmitted as a signal consisting of a pattern of electrical impulses over
the telephone system. The message is the image itself. The information present
within the message depends upon how the image is going to be used.

Question 7

The general block diagram for telecommunication system?

Answer 7

Question 8

The source and transducer for a phone?

Answer 8

“Source” – human being generating the spoken word together with a microphone
(transducer) that converts the sound energy into electrical energy.

Question 9

What is a transmitter?

Answer 9

“Transmitter” – responds to the electrical energy from the microphone and converts it into a form which is suitable for the transmission system, this may include conversion from an analog to digital form, coding and modulation.

Question 10

Explain what is meant by a transmission system?

Answer 10

“Transmission system (includes the channel)” – conveys the electrical energy (signal)
from one end to another, this may constitute something as simple as a single wire, or a
radio channel as in mobile communications, or as complex as a public telephone
network containing fibre-optic or satellite links.

Question 11

Explain what is meant by a receiver?

Answer 11

“Receiver” – accepts the electrical energy from the channel and converts it into a form
ready to be converted back to sound, in essence, it undoes the operations performed
by the transmitter.

Question 12

Explain what is meant by destination(within a speaker system)?

Answer 12

“Destination” – the loudspeaker which converts the electrical energy into sound
energy and the human being who interprets the sound.

Question 13

Define Switching system?

Answer 13

“Switching system” – which connects a certain transmitter to a particular receiver.

Question 14

Define signaling system?

Answer 14

“Signalling system” – which informs the switches which connections should be made.

Question 15

How is compression used?

Answer 15

Files transferred across the Internet are often compressed prior to transmission and
decompressed after transmission.

Question 16

How is compression performed?

Answer 16

Modems apply compression to data leaving or
entering a computer via a phone line.

Question 17

The purpose of compression?

Answer 17

The purpose is to reduce time and resources required for delivery.

Question 18

What is channel bandwidth?

Answer 18

Channel bandwidth is a key resource, namely the range of frequencies available to support the transmitted signal.

Question 19

Redundancy?

Answer 19

Efficient storage and transmission of information in the form of digital data
comes about by removing “redundancy”.

Question 20

Details on significant redundancy?

Answer 20

If a message stream has significant redundancy in that we can often predict what the message will be, then we would expect to be able to transmit the same message with fewer data bits through some form of compression

Question 21

What would happen if we removed all the redundancy?

Answer 21

If we could remove all the redundancy, then the average number of data bits now required would become our numerical definition of the average information in our message stream and, notionally, we would be sending “pure information”.

Question 22

Shannon’s definition of entropy?

Answer 22

Shannon’s definition of entropy, the average information per message, is the key tool for measuring the information and redundancy in a message .

Question 23

What is Shannon’s Fist Theorem?

Answer 23

The coding theorem for noiseless(discrete channels).

Question 24

Shannon’s first theorem for the coding of noiseless discrete channels?

Answer 24

“If a channel is noiseless then all the information transmitted will be recovered by the receiver, hence one can transmit messages from a source with entropy H bits by transmitting on average a minimum of H binary digits per message”

Question 25

The amount of non-redundant information in a message can be quantified by…

Answer 25

The messages entropy

Question 26

What is Huffman coding?

Answer 26

The Huffman coding procedure finds the optimum, uniquely decodable, variable-length code associated with a set of events, given their probabilities

Question 27

The Huffman coding procedure follows a fixed sequence of steps:

Answer 27

• 1. Rank the messages in descending probability
• 2. Combine the 2 message with the lowest probabilities (by summing their probabilities) and insert in the ranked list
• 3. Repeat step 2 until only 2 messages remain
• 4. Assign codewords “0” and “1” to the final 2 messages
• 5. Copy codewords to previous column and append “0” and “1” to uncombined messages
• 6. Repeat step 5 until all message have unique codewords

Question 28

The equation for code efficiency?

Answer 28

Question 29

The equation for line redundancy?

Answer 29

Question 30

Other source coding schemes?

Answer 30

Lempel-Ziv scheme is an example of Universal Coding

Question 31

How does Lempel-Ziv coding operate?

Answer 31

Based upon the idea that an arbitrary string can always be compressed by coding a series of zeros and ones as some previous string (the “prefix string”) together with one new bit

Question 32

How does Lempel-Ziv work?

Answer 32

Question 33

Longest match encoded with triple < o,l,c> where

Answer 33

• o is the offset from the pointer to longest matching string
• l is the length of longest match
• c is the codeword of the next symbol in input string after match

Question 34

For a system or channel with noise (i.e. all practical systems), the capacity C is reduced to…

Answer 34

Question 35

For a modem on an ordinary telephone line S/N = 1000 (30dB), the bandwidth is 3.4 kHz. Therefore, calculate capacity C?

Answer 35

For a modem on an ordinary telephone line S/N = 1000 (30dB), the bandwidth is 3.4 kHz. Therefore,

C = 3400log(1 + 1000) = (3400)(9.97) ≈ 34,000 bps

Question 36

What does the Shanon-Hartley law show?

Answer 36

Question 37

Shannon’s First Theorem:

Answer 37

The coding theorem for noiseless (discrete) channels

“If a channel is noiseless then all the information transmitted will be recovered by the receiver, hence one can transmit messages from a source with entropy H bits by transmitting on average a minimum of H binary digits per message”

Question 38

Shannon’s Second Theorem:

Answer 38

• Coding theory for noisy channels
• “For reliable error free communication we need to transmit on average H/C binary digits per message where H is the message entropy and C is the system or channel capacity (both in bits/symbol)”
• The implication of this is that noise does not limit the reliability of a communication system, only its communication rate
• Note, for reliable communications R must be ≤ C