信息论基础

原文摘录   · · · · · · 

• The initial questions treated by information theory lay in the areas of data compression and transmission. The answers are quantities such as entropy and mutual information, which are functions of the probability distributions that underlie the process of communication. The Kol-mogorov complexity of a binary string is defined as the length of the shortest computer program that prints out the string. Entropy is the uncertainty of a single random variable. A communication channel is a system in which the output depends probabilistically on its input. The entropy H of a random variable is a lower bound on the average length of the shortest description of the random variable. (查看原文)

  Qingcheng 1赞 2014-05-23 09:38:08

• We summarize this by saying, "Almost all events are almost equally suprising." (查看原文)

  OrangeCLK 2013-12-21 03:47:50