Re: why compression doesn't perfectly even out entropy

["Perry E. Metzger" <perry@piermont.com>]

JonWienke@aol.com writes:
> >Is it possible to find a percentage of the key space to eliminate that
> >will optimize security assuming that the attacker will try the easy
> >stuff first (and is it possible to quantify "easy stuff")?
> 
> If you eliminate all repeating byte sequences, such as 00 00 or 7F 7F, you
> will reduce your possible entropy by .07058% (7.99435 bits per byte), and
> eliminate the (astronomically remote) possibility of Hamlet or some other
> English text popping out of your RNG/PRNG.  As long as your key is long
> enough to withstand this slight entropy reduction, you are still OK.

Before making pronouncements like "You are still OK" you ought to
learn a bit more about cryptanalysis. Its tiny little statistical
toeholds like that which permit breaks. I don't know for sure, but my
intuition says that there may very well be instances in which a couple
of little nicks like that into the entropy of a key are sufficient to
radically lower the time to crack something. Since there are far
better techniques available (hash distillation, for instance) for
assuring the quality of a random stream, Jon's suggested techniques
should be regarded as unnecessary and dangerous.

PUBLIC SERVICE ANNOUNCEMENT:

For the benefit of everyone reading, I've become increasingly
convinced that Jon really doesn't understand the topic he's working on
well enough to trust, and he doesn't have the sense to know that he
doesn't understand it well enough. I know enough to know that I'm
extremely ignorant -- he's ignorant enough to think that he knows more
than he does. I don't mean to insult Jon -- I'm sure that in his own
field whatever it is he's a smart enough guy, and he seems like a nice
enough fellow -- but cryptography is a dangerous business -- bad
technique KILLS, literally. Until Mr. Wienke loses his bad case of
hubris I would suggest not taking his technical suggestions.

Perry