Information Theory

• Pictogram: a drawing that resembles the physical object that it represents, eg. a happy face or a human
• Ideogram: a conceptual picture of an abstract idea, like the symbol of peace
• Rebus theory: sound + sound = new meaning
• Proto language: instead of letters using icons

Morse code

• bits and dashes to compose letters
• send through electric impulses
• uses the binary combinations like the flag system
• the most used letters were the easier to type

information transmission

Channel Capacity

• Symbol rate (n): how many symbols transferred per second eg. a bitrate. Also known as Baud.
• Difference (s): how many differences per symbol, eg. a bit would be two differences per symbol.
• the symbol rate would dictate the depth of the decision tree
• Message space: the width of the base of the decision tree or $s^n$. In other words, how many different symbols or letters can be sent in a specific amount of time.

Measure amount of information transmitted in a unit of time by

$log(s^n)$