Welcome to our site! EDAboard.com is an international Electronics Discussion Forum focused on EDA software, circuits, schematics, books, theory, papers, asic, pld, 8051, DSP, Network, RF, Analog Design, PCB, Service Manuals... and a whole lot more! To participate you need to register. Registration is free. Click here to register now.
You know, entropy means an uncertainty having bits unit in random variable. Thus it depends on the distribution(probability) of random variable, not an actual value of random variable. Especially, the probability in an entropy is probability mass function, that is a discrete case.
In other words, when we use continuous probability function, that is differential entropy.
When it comes to entropy rate, it is applicable to stochastic function. Not only one stochastic function, but also mutiple stochastic functions has the entropy. However, as the number of the functions increase, the entropy will vary. That is the rate of entropy.
:::
A measure of how much information that can be
transmitted and received with a negligible probability
of error is called the channel capacity.
To determine this measure of channel potential, assume that a channel
encoder receives a source symbol every Ts second.
With an optimal source code, the average code length of all source symbols is equal to the entropy rate of the source.
If S represents the set of all source symbols and the entropy rate of the source is written as H(S),
the channel encoder will receive on average H(S)/ Ts information bits per second
"""
above lines are from a paper, what I dont understand is that
Entropy rate of the source H(s) means information contained in one transmitted symbol or
The information contained in all of possible source symbols a source has.
:::
A measure of how much information that can be
transmitted and received with a negligible probability
of error is called the channel capacity.
To determine this measure of channel potential, assume that a channel
encoder receives a source symbol every Ts second.
With an optimal source code, the average code length of all source symbols is equal to the entropy rate of the source.
If S represents the set of all source symbols and the entropy rate of the source is written as H(S),
the channel encoder will receive on average H(S)/ Ts information bits per second
"""
above lines are from a paper, what I dont understand is that
Entropy rate of the source H(s) means information contained in one transmitted symbol or
The information contained in all of possible source symbols a source has.
You better refer "Elements of information theory".
In the theorem 5.4.2, the minimum expected codeword length per symbol is described. Consequently, H(s) mentioned above can be expressed in detail as like, H(S) = H(s1,s2, ... ,sn)/n. Thus I can say that it is the rate per one symbol but all information.
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.
