Syllabus
|
Course Code: EC-307 Course Name: Information Theory and Coding |
||
| MODULE NO / UNIT | COURSE SYLLABUS CONTENTS OF MODULE | NOTES |
|---|---|---|
| 1 | Probability, random variables, Probability distribution functions and probability density functions, Expectation, moments, Random Processes, mean and Auto Correlation, Stationary and ergodicity, Information theory : the definition of information, the zero-memory information source, entropy for discrete ensembles; properties of entropy, Shannon's noiseless coding theorem; Encoding of discrete sources, | |
| 2 | Properties of codes: Introduction, types of codes: uniquely decodable codes, instantaneous codes, construction of an instantaneous code, Kraft inequality: statement and discussion and Proof, Markov sources; Shannon's noisy coding theorem and converse for discrete channels; Calculation of channel capacity and bounds for discrete channels; Application to continuous channels. | |
| 3 | Coding information sources: The average length of a code, Shannon’s First Theorem, Finding binary compact codes- Huffman codes, Code efficiency and redundancy; Channels and mutual information: Information channels, Binary symmetric channels, Probability relations in a channel, A priori and A posteriori entropies, Mutual information, properties of mutual information, types of channels: Noiseless, deterministic, Cascaded channels, Channel capacity. | |
| 4 | Channel Coding: Shannon second theorem for Noisy channels, Introduction to error control coding, Types of codes, Maximum Likelihood decoding, Linear block codes, Error detecting and correcting capabilities of a block code, Hamming code, cyclic code, convolutional arithmetic codes. | |
