By the end of the course, the students will be able to

1. Comprehend how to quantify information and uncertainty.

1.1 Calculate the information content of a random variable as generated from a random experiment based on its probability distribution.

1.2 Measure the information content of multiple random entities.

1.3 Determine the statistical relation between random variables based on mutual information.

1.4 Utilize identities and inequalities commonly used in information theory.

1.5 Analyze the characteristics of typical sequences based on information theoretic measures.

2. Apply concepts of information theory to the data compression problem.

2.1 Sketch the proof regarding the limits of data compression.

2.2 Design efficient data compression schemes for a given information source.

2.3 Interpret performance of a data compression scheme based on source coding theorems.

2.4 Differentiate between lossless and lossy source coding techniques and their performance limits.

3. Apply information theory to noisy communication channels.

3.1 Define channel capacities and properties.

3.2 Sketch the proof regarding the limits of error-free communication.

3.3 Understand the impact of channel coding on noisy communication.

3.4 Calculate the capacity of communication channels.

3.5 Generalize the results for the discrete channels to continuous channels and signals.