[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Snake Oil FAQ 1.0
Obviously, there's still work to be done, but that's why there are
numbers higher than 1.0, right? :-)
-matt
URL: http://www.research.megasoft.com/people/cmcurtin/snake-oil-faq.html
Version: 1.0
Archive-name: cryptography-faq/snake-oil
Posting-Frequency: monthly
Snake-Oil Warning Signs
Encryption Software to Avoid
Copyright � 1996 Matt Curtin
$Id: snake-oil-faq.html,v 1.0 1996/09/27 21:15:04 cmcurtin Exp $
Distribution
Distribution of this document is unlimited. We're specifically interested in
reaching people making decisions about what sorts of crypto to use (if any
at all), both for their organizations and for themselves, especially those
who are non-experts in the field of cryptography and security. This is a
work-in-progress. Feedback is greatly appreciated.
The Snake Oil FAQ is posted monthly to cypherpunks, sci.crypt, alt.security,
comp.security, comp.answers, and comp.infosystems.
Disclaimer
All contributors' employers will no doubt disown any statements herein.
We're not speaking for anyone but ourselves. This is a compilation things
that are common among snake oil vendors. It cannot be the sole metric by
which a security product is rated, since there can be exceptions to most (or
all?) of these rules. (But if you're looking at something that sounds
familiar on several of the 'things to watch out for,' you're probably
dealing with snake oil. From time to time, a reputable and decent vendor
will produce something that is actually quite good, but will use some
braindead marketing technique, so be aware that exceptions to general rules
can exist.)
Every effort has been made to produce an accurate and useful document, but
the information contained herein is completely without warranty. If you find
any errors, or wish to otherwise contribute, please contact the document
keeper, Matt Curtin <[email protected]>
History
With the rise in the number of crypto products becoming available came a
rise in the amount of ineffective or outright bogus products. After some
discussion about this on the cypherpunks list, Robert Rothenburg
<[email protected]> wrote the first iteration of the Snake Oil FAQ. Matt
Curtin took the early text and munged it into its current state with the
help of the listed contributors (and probably some others whose names have
inadvertently missed. Sorry in advance, if this is the case.)
Introduction
This really isn't much of a "FAQ" in the sense that one generally expects to
see them: in a question and answer format. Perhaps it will be rewritten as
such in the future, but currently, it is more traditionally-formatted paper
that covers many topics that are the subject of frequently asked questions.
Good cryptography is an excellent and necessary tool for almost anyone.
However, there is a multitude of products around. Many good cryptographic
products are available, both commercial (including shareware) and free.
However, there are also some extremely bad cryptographic products (known in
the field as "Snake Oil"), which not only fail do their job of providing
security, but are based on, and add to, the many misconceptions and
misunderstandings surrounding cryptography and security.
Why "snake oil?" The term is used in many fields to denote something that is
sold without consideration of its quality, or its ability to live up to
claims made by its vendor. This term originally applied to that sold in
traveling medicine shows, where the salesmen would claim their elixer would
cure just about any ailment that a potential customer could have. Listening
to some of the claims made some by modern day crypto vendors, "snake oil" is
a surprisingly apt name.
Superficially, it is difficult for someone to distinguish the output of a
secure encryption utility from snake oil: both look garbled. The purpose of
this document is to present some obvious "red flags" that people unfamiliar
with the nuts and bolts of cryptography can use as a guideline for
determining whether they're dealing with snake oil or the Real Thing.
For a variety of reasons, this document is general in scope and does not
mention specific products or algorithms as being "good" or "Snake Oil".
When evaluating any product, be sure to understand what your needs are. For
data security products, what do you need protected? Do you want an archiver
that supports strong encryption? An E-mail client? Something that will
encrypt on-line communications? Do you want to encrypt an entire disk or
partition, or selectively some files? How secure is "secure enough?" Does
the data need to be unreadable by third parties for 5 minutes? One year? 50
years? 100 years? Is the third party someone's kid sister? An individual? A
corporation? A government?
Beware of products that are designed for a specific task (such as data
archiving, for example), and add encryption in as an additional feature.
Typically, it's better to use an encryption utility for encryption, rather
than some tool designed for another purpose that adds encryption to its list
of features.
Some basics
The cryptography-faq (found at
http://www.cis.ohio-state.edu/hypertext/faq/usenet/cryptography-faq/top.html)
is a more general tutorial of cryptography, and should also be consulted. In
an effort to make this FAQ more complete, some very basic topics are
included below.
Conventional vs. Public Key Cryptography
There are two basic types of cryptosystems: symmetric (also known as
"conventional," sometimes also called "secret key") and asymmetric ("public
key.") Symmetric ciphers require both the sender and the recipient to have
the same key. That key is applied to encrypt the data by the sender, and
again by the recipient to decrypt the data. The problem here is getting the
sender and recipient to share the key. Asymmetric ciphers are much more
flexible, from a key management perspective. Each user has a pair of keys: a
public key and a private key. The public key is shared widely, given to
everyone, while the private key is kept secret. If Alice wishes to mail Bob
some secrets, she simply gets (and verifies!) Bob's public key, encrypts her
message with it, and sends it off to Bob. When Bob gets the message, he uses
his private key to decrypt the message.
Secrecy vs Integrity: What are you trying to protect?
For many users of computer based crypto, preserving the contents of a
message is as important as as protecting its secrecy. Damage caused by a
modified message can often be worse than that caused by its disclosure. For
example, it may be disquieting to discover that a hacker has read the
contents of your funds transfer authorization, but it's a disaster for him
to change the transfer destination to his own account.
Encryption by itself does not protect a message from change. In fact, there
are several techniques for changing the contents of an encrypted message
without ever figuring out the encryption key. If the integrity of your
messages is important, don't just rely on secrecy to protect them. Check the
vendor's claims for an explanation of how their product protects the message
from undetected modification.
The verification of public keys is an important step. Failure to verify
Bob's public key leaves open the possibility that Alice is sending her
secrets to someone else, who simply claims to be Bob, using a key that has
Bob's name on it, but whose associated private key is in the hands of an
attacker.
Asymmetric ciphers are much slower than their symmetric counterparts. Also,
key sizes must be much larger. See the cryptography FAQ for a more detailed
discussion of these topics.
Key Sizes
Some ciphers, while currently secure against most attacks, are not
considered viable in the next few years because of relatively small key
sizes and increasing processor speeds (making a brute-force attacks - trying
every possible key - feasible). The tables below should give some general
guidelines for making intelligent decisions about the key length you need.
If the key is too short, the system will be easily broken, even if the
cipher is a good one.
Having stated the above, it is important to note that a common feature of
snake oil is to have large keys. Often, the claimed key lengths are much
longer than what is practical, usually due to the vendor's confusion between
symmetric and asymmetric cipher key length requirements. (For example, a
vendor who claims to use a strong symmetric cipher with a 2048 bit key is
probably lacking some basic understanding of key length requirements, and
requisite computing power for performing various functions with the keys in
question.)
In [1] and [2], we're presented with some guidelines for deciding
appropriate key length. (It is important to note that this is based on the
ability to predict computing power 40, 65, and 100 years from now. Major
breakthroughs in computing power 30 years from now might render everything
on this chart kiddieplay. This is included so the reader will be able to get
a reasonable idea of symmetric key length requirements, and have some sort
of a guideline for determining whether the key length of the product he's
interested in even makes sense.) The following chart appears in [1].
Security Requirements for Different Information
Type of Traffic Lifetime Minimum [Symmetric]
Key Length
Tactical military information minutes/hours 56-64 bits
Product announcements, mergers, interest
rates days/weeks 64 bits
Long-term business plans years 64 bits
Trade secrets (e.g., recipe for
Coca-Cola) decades 112 bits
H-bomb secrets >40 years 128 bits
Identities of spies >50 years 128 bits
Personal affairs >50 years 128 bits
Diplomatic embarrassments >65 years at least 128 bits
U.S. Census data 100 years at least 128 bits
As mentioned earlier, asymmetric ciphers require significantly longer keys
to provide the same level of security as their symmetric cipher
counterparts. Here is a comparison table, again, from [1]. (Due to
differences between symmetric and asymmetric algorithms, key length
comparisons between the two is difficult. The following is intended to give
the reader just a general idea of what is roughly comparable, in order to be
able to weed out claims of security of, for example, ciphers with 100-bit
asymmetric keys.)
Symmetric and Public-Key Lengths With
Similar Resistance to Brute-Force Attacks*
Symmetric Key Length Public-key Key Length
56 bits 384 bits
64 bits 512 bits
80 bits 768 bits
112 bits 1792 bits
128 bits 2304 bits
*These key sizes are for public key cryptosystems based on the problem of
factoring large integers, and apply to a number of ciphers based on the
discrete log problem (difficulty of taking logarithms in a finite field.) A
variation of the discrete log problem (known as Elliptic Curve Discrete
Logarithm Problem), where the cryptosystem is based on computations on
points of an elliptic curve over a finite field, for example, has been shown
to be resistant to brute-force attacks with much smaller keys than other
discrete log problem-based ciphers. Ciphers based different problems have
different key size requirements. Each type of algorithm's key size
requirements depend on the mathematical problem on which the system is
based. So, it's important to find out what algorithm (or at least
mathematical problem the algorithm uses) and key size is used. One without
the other is meaningless.
Implementation Environment
Other factors that can influence the relative security of a product are
related to its environment. For example, in software-based encryption
packages, is there any plaintext that's written to disk (perhaps in
temporary files)? What about operating systems that have the ability to swap
processes out of memory on to disk? When something to be encrypted has its
plaintext counterpart deleted, is the extent of its deletion a standard
removal of its name from the directory contents, or has it been written
over? If it's been written over, how well has it been written over? Is that
level of security an issue for you? Are you storing cryptographic keys on a
multi-user machine? The likelihood of having your keys illicitly accessed is
much higher, if so. It's important to consider such things when trying to
decide how secure something you implement is (or isn't) going to be.
Some Common Snake-Oil Warning Signs
The following are some of the "red flags" one should watch for when
examining an encryption product
* Technobabble
The vendor's description of the product may contain a lot of
hard-to-follow use of technical terms to describe how the product
works. If this appears to be confusing nonsense, it may very well be
(even to someone familiar with the terminology). Technobabble is a good
means of confusing a potential user and masking the fact that the
vendor doesn't understand anything either.
A sign of technobabble is a description which drops a lot of technical
terms for how the system works without actually explaining how it
works. Often specifically coined terms are used to describe the scheme
which are not found in literature about cryptology.
Further, if the marketing material isn't clear, what reason is there to
believe that the instructions are any better? Even the greatest of
products, if not used properly, can be rendered useless. If you can't
understand what a vendor is saying, you're most likely better off
finding something that makes more sense.
* New Type of Cryptography?
Beware of any vendor who claims to have invented a "new type of
cryptography" or a "revolutionary breakthrough". Truly "new
breakthroughs" are likely to show up in the research literature, and
professionals in the field are typically won't trust them until after
years of analysis, by which time they are not so new anymore.
Avoid software which claims to use 'new paradigms' of computing such as
cellular automata, neural nets, genetic algorithms, chaos theory, etc.
Just because software uses a different method of computation doesn't
make it more secure. (As a matter of fact, these techniques are the
subject of ongoing cryptographic research and nobody has published
successful results based on their use yet.)
Anything whose authors claim to have invented a new public key
cryptosystem without publishing the details or underlying mathematical
principles is highly suspect. Modern cryptography is grounded in
mathematical theory. The security is based on problems that are known
(or widely believed) to be hard to solve.
It's important to understand the difference between a new algorithm or
cipher and a new product. Engaging in the practice of developing
ciphers and cryptographic products is a fine thing to do. However, to
do both, at the same time, is foolish. Many snake oil vendors brag
about how they do this, despite the lack of wisdom in such activity.
The strength of any encryption scheme is only proven by the test of
time. New crypto is like new pharmaceuticals, not new cars. In some
ways, though, it's worse: if some pharmaceutical company has some bogus
stuff out there, people will start getting really sick. If you're using
bogus crypto, you likely won't have any idea that your secrets aren't
as secret as you think.
* Secret Algorithms
Avoid software which uses secret algorithms. Security through obscurity
is not considered a safe means of protecting your data. If the vendor
does not feel confident that the method used can withstand years of
scrutiny by the academic and professional crypto community, then you
should be wary of trusting it. (Note that a vendor who specializes in
cryptography may have a proprietary algorithm which they'll show to
others if they sign a non-disclosure agreement. If the vendor is
well-reputed in the field, this can be an exception. On the other hand,
if you don't know which vendors are and aren't reputable, you can't
take their words for it. You're typically best off avoiding that which
is secret.)
Beware of specially modified versions of well-known algorithms. This
may intentionally or unintentionally weaken the cipher.
The use of a trusted algorithm, with technical notes explaining the
implementation (or better yet, availability of the source code for the
product itself) are signs that a vendor is confident about their
product's security. You can take the implementation apart and test it
yourself. A lock where attackers can see the internal mechanisms, and
still not be able to break it is a strong lock, indeed.
A common excuse for not disclosing how a program works is that "hackers
might try to crack the program's security." While this may be a valid
concern, it should be noted that such 'hackers' can reverse engineer
the program to see how it works anyway. If the program is implemented
properly and the algorithm is secure, this is not a problem. (If a
hypothetical 'hacker' was able to get access you your system, access to
encrypted data might be the least of your problems.)
* Experienced Security Experts and Rave Reviews
Beware of any product claiming that "experienced security experts" have
analyzed it, but it won't say who (especially if the scheme has not
been published in a reputable journal).
Don't rely on reviews in newspapers, magazines or television shows,
since they generally don't have cryptologists (celebrity hackers who
know about telephone systems don't count) to take the software apart
for them.
Just because the vendor is a well known company or the algorithm is
patented doesn't make it secure either.
* Unbreakability
Some vendors will claim their software is "unbreakable". This is
marketing hype, and a common sign of snake-oil. Avoid any vendor that
makes unrealistic claims. (If it sounds too good to be true, it
probably is.)
No algorithm is unbreakable. Even the best algorithms are breakable
using "brute force" (trying every possible key), but if the key size is
large enough, this is impractical even with vast amounts of computing
power.
One-time pads are unbreakable, but they must be implemented perfectly,
which is, at best, very difficult. See the next section for a more
detailed discussion.
Some companies that claim "unbreakability" actually have serious
reasons for saying so. Unfortunately, these reasons will generally turn
out to depend on some narrow definition of what it means to "break"
their security. For example, true one time pads are technically
"unbreakable" as far as secrecy goes, but only if several difficult and
important conditions also hold. Even then, they are trivially
vulnerable to known plaintext attacks on the message's integrity. Other
systems may be "unbreakable" only until one of the communicating
devices (a laptop, for example) is stolen.
So, be sure to find out exactly what the "unbreakable" properties of
the system are, and decide if the more breakable portions also provide
adequate security. Often, less experienced vendor representatives will
roll their eyes and say, "Of course it's not unbreakable if you do
such-and-such." The point is that the exact nature of "such and such"
will vary from one product to another. Pick the one that matches your
operational needs the best.
* One-Time-Pads
A vendor might claim the system uses a one-time-pad (OTP), which is
theoretically unbreakable. (Technically, OTP-generated ciphertext has
an equal chance of being each possible plaintext. For example, "598v *$
_+~xCtMB0" has equal probabilities of decrypting to "the whole year
in", "the hole youre in", and "you are a weenie!") Snake-oil sellers
will try to capitalize on the known strength of an OTP. It is important
to understand that any variation in the implementation (which is often
done to get around the inherent key management problems of OTPs) means
that it is not an OTP, and has nowhere near the security of an OTP.
An OTP system is not an algorithm. It works by having a "pad" (called
such because originally paper pads were used, before general-purpose
computers came into being) of random bits in the possession of both the
sender and recipient, but absolutely no one else. (The pad must be sent
from one to the other securely, such as in a locked briefcase
handcuffed to the carrier, and that sort of thing.) The message is
encrypted using the next n bits in the pad as they key, where n is the
number of bits in the message. After the bits are used from the pad,
they're destroyed, and can never again be used. The bits in the pad
must be truly random, generated using a real random source, such as
specialized hardware, radioactive decay timings, etc., and not from an
algorithm or cipher. Anything else is not a one-time-pad. Further, if
the keys (i.e., random bit "pads") are provided by the vendor, the
quality of these cannot be verified. How do you know that they aren't
sending the same bits (or some trivial mutation thereof) to everyone?
Or keeping a copy for themselves? Or selling a copy to your competitors
or enemies?
OTPs are highly impractical for general purpose cryptography, since the
need for random bits is very high, and key management is so cumbersome.
OTPs are only practical for extremely low bandwidth communication
channels where two parties have the means to exchange pads through a
different method from that of their messages. (It is rumored that a
link from Washington, D.C., to Moscow was (is?) encrypted with an OTP.)
A lesson from the VENONA project (see NSA's web site) is that OTPs are
seriously vulnerable if a pad is ever reused. It does not take the
resources of a government agency to crack a reused pad. Therefore, the
real limitation to their practical use is the generation and
distribution of truly random keys for them. You have to distribute at
least one bit of key for every bit of data transmitted, including any
encrypted protocol data that's sent. If you reuse your pads you run the
risk of compromising all data sent with the reused pad.
The vendor might (or might try to) confuse random session keys or
initialization vectors with OTPs.
* Algorithm or product XXX is insecure
Be wary of anything that makes claims that particular algorithms or
other products are insecure without backing up those claims (or at
least citing references to them).
Sometimes attacks are theoretical or impractical (requiring special
circumstances or massive computing power running for many years), and
it's easy to confuse a layman by mentioning these. These usually
involve either trying every possible combination of bits for form keys,
and trying every possible key until a solution is found, factoring
large numbers, or some other cryptanalysis that's just as
computationally intensive as one of these methods.
* Keys and Passwords
The "key" and the "password" are not the same thing. The "key"
generally refers to the actual data used by the cipher, while the
"password" refers to the word or phrase the user types in, which the
software converts into the key (usually through a process called
"hashing" or "key initialization").
The reason this is done is because the characters a user is likely to
type in do not cover the full range of possible characters. (Such keys
would be more redundant and easier for an attacker to guess.) By
hashing a key can be made from an arbitrary password that covers the
full range of possible keys. It also allows one to use longer words, or
phrases and whole sentences as a "passphrase", which is more secure.
If the system limits the size of the key or passphrase to something
that seems too low, it probably is. If the actual "password" is the
cipher's key (rather than hashing it into a key, as explained above),
avoid it.
If the vendor confuses the distinctions between bits, bytes and
characters when discussing the key, avoid this product.
Convenience is nice, but be wary of anything that puts too much
emphasis on ease of use, without due consideration to cryptographic
strength. Avoid anything that lets anyone with your copy of the
software to access files, data, etc. without having to use some sort of
key or passphrase.
Avoid anything that doesn't let you generate your own keys (ie, the
vendor sends you a key in the mail, or it's embedded in the copy of the
software you buy).
Avoid anything by a vendor who does not seem to understand the
difference between public-key (asymmetric) cryptography and secret-key
(symmetric) cryptography.
* Lost keys and passwords
If the vendor (or a third party) claims it can recover lost passwords
(without using a key-backup or escrow feature), avoid it: a flaw is
obviously present, and used to retrieve the contents of an encrypted
message.
If there is a key-backup or escrow feature, are you in control of the
backup, or does the vendor or someone else hold a copy of the key? (Is
someone else able to recover your key as easily as you can?) Remember,
you have no security against someone who has your key.
* Exportable from the USA
If the software is made in the US, can it be exported? If the answer is
yes, chances are it's not very strong. Strong cryptography is
considered munitions in terms of export from the United States, and
requires approval from the State Department. Chances are if the
software is exportable, the algorithm is weak or it is crackable (hence
it was approved for export).
If the vendor is unaware of export restrictions, avoid the software:
the vendor is not familiar with the state of the art. (For example, if
someone claims that the IDEA cipher is exportable from the US, while
most other vendors (or the State Department!) do not make such
assertions, they're probably lacking sufficient clue to provide you
with strong cryptographic software.)
Because of export restrictions, some legitimate (not-Snake Oil)
products may have a freely exportable version for outside of the USA,
which is different from a separate US/Canada-only distribution. (Of
course, a freely exportable version isn't secure, since it probably
just uses a much smaller key, one that could be easily broken.) Also
note that just because software has made it outside of the US does not
mean that it is exportable: sometimes a utility will be illegally
exported and posted on an overseas site. There are no restrictions on
importing crypto products into the US, so a foreign vendor can legally
offer a single, secure version of a product for the entire world.
* "Military Grade" Encryption
Many crypto vendors claim their solution is "military grade." This is a
term with no real meaning, since there isn't a real metric by which
something can be judged "military grade," except for it to be actually
used by various armed forces. Since they don't reveal what they're
using, it's neither possible to prove nor to disprove something as
being "military grade." Some good crypto products unfortunately also
use this term. (Watch for this one especially in combination with other
snake oil indicators, i.e., "our military grade encryption system is
exportable from the US!")
Other Considerations
Interface isn't everything: user-friendliness is an important factor, but if
the product isn't secure then you're better off with something that is
secure (if not as easy to use).
No product is secure if it's not used properly. You can be the weakest link
in the chain if you use a product carelessly. Do not trust any product to be
foolproof, and be wary of any product that claims it is.
Glossary
algorithm A procedure or mathematical formula. Cryptographic
algorithms convert plaintext to and from ciphertext.
cipher Synonym for "cryptographic algorithm"
cryptanalysis To solve or "break" a cryptosystem.
escrow A third party able to decrypt messages sent from one
person to another. Although this term is often used in
connection with the US Government's "Clipper" proposals,
it isn't limited to government-mandated ability to access
encrypted information at will. Some corporations might
wish to have their employees use cryptosystems with escrow
features when conducting the company's business, so the
information can be retrieved should the employee be unable
to unlock it himself later, (if he were to forget his
passphrase, suddenly quit, get run over by a bus, etc.)
Or, someone might wish his spouse or lawyer to be able to
recover encrypted data, etc., in which case he could use a
cryptosystem with an escrow feature.
initialization One of the problems with encrypting such things as files
vector in specific formats (i.e., that of a word processor,
email, etc.) is that there is a high degree of
predictability about the first bytes of the message. This
could be used to break the encrypted message easier than
by brute force. In ciphers where one block of data is used
to influence the ciphertext of the next (such as CBC), a
random block of data is encrypted and used as the first
block of the encrypted message, resulting in a less
predictable ciphertext message. This random block is known
as the initialization vector. The decryption process also
performs the function of removing the first block,
resulting in the original plaintext.
ITAR International Traffic in Arms Regulations. These are the
rules by which munitions (including cryptography), as
defined by the US State Department, may (or may not) be
exported from the US.
key A piece of data that, when fed to an algorithm along with
ciphertext, will yield plaintext. (Or, when fed to an
algorithm along with plaintext, will yield ciphertext.
random session This is a temporary key that is generated specifically for
key one message. Typically, in public key cryptosystems, the
message to be sent is encrypted with a symmetric key that
was specifically generated for that message. The encrypted
version of that message, as well as the associated session
key can then be encrypted with the recipient's public key.
When the recipient decrypts the message, then, the system
will actually decrypt the message it gets (which is the
ciphertext message and the symmetric key to decrypt it),
and then use the symmetric key to decrypt the ciphertext.
The result is the plaintext message. This is often done
because of the tremendous difference in the speed of
symmetric vs. asymmetric ciphers.
Contributors
The following folks have contributed to this FAQ.
Jeremey Barrett <[email protected]>
Gary Ellison <[email protected]>
<[email protected]>
Larry Kilgallen <[email protected]>
Dutra Lacerda <[email protected]>
<[email protected]>
Jim Ray <[email protected]>
Terry Ritter <[email protected]>
Robert Rothenburg <[email protected]>
Adam Shostack <[email protected]>
Rick Smith <[email protected]>
Randall Williams <[email protected]>
Jim Ray <[email protected]>
References
1. B. Schneier, Applied Cryptography, second edition, John Wiley & Sons,
1996
2. M. Blaze, W. Diffie, R. L. Rivest, B. Schneier, T. Shimomura, E.
Thompson, M. Wiener, "Minimal Key Lengths for Symmetric Ciphers to
Provide Adequate Commercial Security," available from
ftp://ftp.research.att.com/dist/mab/keylength.ps
--
C Matthew Curtin MEGASOFT, INC Chief Scientist
I speak only for myself. Don't whine to anyone but me about anything I say.
Hacker Security Firewall Crypto PGP Privacy Unix Perl Java Internet Intranet
[email protected] http://research.megasoft.com/people/cmcurtin/