Syllabus
|
Course Code: ST-403 & ST-404 Course Name: (iv) Information Theory |
||
| MODULE NO / UNIT | COURSE SYLLABUS CONTENTS OF MODULE | NOTES |
|---|---|---|
| 1 | Introduction : communication process, communication system, measure of information, unit of information. Memory less finite scheme: Measure of uncertainty and its properties, sources and binary sources. Measure of information for two dimensional discrete finite probability scheme: conditional entropies, Noise characteristics of a channel, Relations among different entropies. | |
| 2 | Measure of Mutual information, Shanan's fundamental inequalities, Redundancy, Efficiency and channel capacity, capacity of channel with symmetric noise structures, BSC and BEC, capacity of binary channels, Binary pulse width communication channel, Uniqueness of entropy function. | |
| 3 | Elements of encoding : separable binary codes, Shannon-Fano encoding, Necessary and sufficient conditions for noiseless coding. Theorem of decodibility, Average length .of encoded messages; Shannon's Binary Encoding. | |
| 4 | Fundamental theorem.of discrete noiseless encoding, Huffman's minimum redundancy code, Gilbert-Moore encoding. Error detecting and Error correcting codes, Geometry of binary codes, Hammings single error correcting code. | |
