Netscape's RNG

[Ian Goldberg <iagoldbe@csclub.uwaterloo.ca>]

Someone on the list (sorry, I forget who), suggested that Netscape's RNG
be looked at to see if the secret part of the SSL RC4/40 master key could
be determined more directly.

I used gdb on the Solaris version.  SSL_GenerateRandomBytes() is called
when random bytes are needed.  It initializes the RNG, if necessary, and then
calls RNG_GenerateRandomBytes.

When I run "netscape https://banking.wellsfargo.com/",
SSL_GenerateRandomBytes() is called 3 times; the first time, 32 bytes
are produced (I don't know what they're for); the second time, 16 bytes
are produced (the Challenge data); the third time, 64 bytes are produced,
the first 16 of which are the master key (the first 11 of which are sent
in the clear, and the next 5 are our goal).

Here's my own hand-reverse-assembly of RNG_GenerateRandomBytes:
(Correctness not actually guaranteed in any way...)

-----8<-----8<-----
struct RNG
{
    unsigned char md5bytes[0x10];
    unsigned char randbytes[0x10];
    int size;
    void *md5data;
};

RNG_GenerateRandomBytes(struct RNG *i0, char *i1, int i2)
{
    char buf[0x20];
    int o1,o2;

    while (i2 > i0->size)
    {
	memcpy(i1, &(i0->randbytes)+0x10-i0->size, i0->size);
	i1 += i0->size;
	i2 -= io->size;
	if (err = MD5_Begin(i0->md5data)) return err;
	if (err = MD5_Update(i0->md5data, &(i0->md5bytes), 0x10)) return err;
	if (err = MD5_End(i0->md5data, &(i0->randbytes), buf, 0x10)) return err;
	i0->size = 0x10;
	o2 = 0;
	o1 = &(i0->md5bytes[0x0f]);
	do
	{
	    if ((*o1)++) break;
	    --o1;
	} while (++o2 <= 0x0f);
    }

    /* i2 <= i0->size */
    memcpy(i1, &(i0->randbytes)+0x10-i0->size, i2);
    i0->size -= i2;
    return 0;
}
-----8<-----8<-----

It looks like that Compilers course came in handy...

So it's not linear congruential.  I guess the next step is to figure out
how it's seeded, but that not for me to do (at least not tonight...).

Here's another question about a more direct method:
The 5 secret bytes are encrypted with the server's public (RSA?) key.
Does the server use the same public key every time?  How do you read
the public key, given the Certificate Data (what's the format of the
certificate)?  Is it feasible to try to attack the public key with a
massively parallel (Internet) factoring program (a la RSA-129)?
Assuming that the modulus is _big_, it still is worthy to note that,
unlike cracking individual challenges, cracking the public key will
compromise _all_ communications with that server (until they catch on
and pay $$$ for another key (I think?)).

Just some thoughts,

   - Ian "it's only 10pm _here_, but it's 2am in Nova Scotia!"