Re: why compression doesn't perfectly even out entropy

["Perry E. Metzger" <perry@piermont.com>]

Hal writes:
> The first is whether this mysterious black box, the entropy estimator,
> is really possible.  In practice the only way to know how much entropy
> you've gotten is to have a model for how the data is being generated,
> and to deduce from that an estimate of the entropy rate.  So the entropy
> estimator can't be a general-purpose calcluation, but it must be one
> which is specifically chosen, developed and tuned for the specific source
> of entropy you are dealing with.

I couldn't possibly say that better. Its the central point.

> So I think the lesson is that there is only one way to estimate entropy,
> and that is to study your source.  I have to agree with Perry that this
> filtering concept is not the way to go.  It is a red herring that lures
> you in the direction of automatic entropy estimation, and that is really
> not safe.

Thank you; you are making the point far better than I did.

.pm