RATINGS: proposal

[hughes@ah.com (Eric Hughes)]

>I'm not sure that I understand why a mail
>message could be rated multiple times by the same rator, unless you
>mean that one might define "axes of rating", like "content",
>"spelling", "novelty", etc.  

This is exactly the reason.

We've already discussed saliency.  There are a few more criteria I can
think of immediately (including the one we know):

-- salience.  What is the article about?

-- clarity.  In the age of information overload, clarity and brevity
are the soul of politeness.  Consider this.  When you post to
cypherpunks, several hundred people may read your message.  If you can
spend one minute making your words clear, you will save hours in
aggregate for all involved.  But in fact, if it's not clear enough, I
don't want to read it at all, saving even more time in aggregate.

Example of a characteristically low clarity rating: L. Detweiler

-- novelty.  Repeated arguments have as their primary quality that
they are ... repeated.  Do I want the same rehash over and over?  How
many times do I want to hear about hidden trapdoors in DES?  Zero.

Example of a characteristically low novelty rating: Sternlight

These two examples are not hypothetical.

-- fact/query/opinion.  What is the balance between verifiable claims
of fact, question or request for help or information, and mere
assertion?  People who wish to help newbies should be able to do so,
and those who wish to ignore them should be able to do that.

-- readware.  A fellow at Bell Labs is working with 'readware', which
is a computer analog of the smudged edges of a reference book in the
place where it's opened to most.  A simple readware scheme could
deliver the number of lines that were read before the article was
deleted.  This information is pretty easy to collect, and requires
almost no user intervention.

>I think that such a scheme is good, but
>is starting to place more load on the rator.

Each rater need not be required to publish a full rating, nor even
rate each article.  No one is supposed to rate an article, and anybody
should be capable of it.

>[proposes an email-header based syntax for a rating]

>The PGP information is intended to facilitate "rating reputations" so
>that MUAs could be configured to "trust" ratings from people with good
>reputations for rating in ways that meet the user's idea of "goodness."

Certainly the ratings format should allow for digital signatures.  The
identity of the rater is certainly relevant to a decision process.

One of the immediate reasons for this is that one might easily want
one's ratings to be private, and yet participate publically.  Here is
a use for pseudonyms that an ordinary person can understand.  If you
don't want someone to know that you think badly of them, don't tell
them.  But you can tell the world under a pseudonym.  It's like an
anonymous referee.

>[on 0-100 scale]
>the common case (50) is the MOST unlikely to motivate someone to issue
>a rating message.  I'm not sure what to do about this problem.

The Central Limit Theorem comes to the rescue.  It says that if you
add together enough instances of random variables with the same
distribution, you always get a Gaussian distribution (a bell curve).

[ An aside.  This is the secret reason that statistical mechanics
works.  Add up enough atoms, and you _can_ assume a Gaussian.  My
physics professor did not tell me this.  Grr. ]

Get enough raters, and the ratings can be first-approximated to good
accuracy by the mean and variance.  High variance means it's
controversial, sometimes a positive characteristic in its own right.
And if you get a bimodal distribution, so much the more.

>X-Mail-Rating: cypherpunks

Certainly a list identifier for mail handling would be useful, but
that's not part of a rating syntax.

>1) Heavy dependance on Message-Id: field of messages and not all
>messages bear one of these.

You check.  Every single one from toad.com does.  Message-Id is a
required field.  If mail doesn't have it, the mailer is misconfigured.
What most mailers do is that if they don't see a Message-Id, they add
their own; this is what toad.com does.

>2) This scheme rewards people who wait on the mail message ratings to
>come in then read the mailing list.  

That is the idea.  Some people want to read everything, some don't.
Those who read early will tend to get their own words read more often,
and this may be reason enough to rate.  A good reputation for rating
may also translate into a good reputation for writing.

>(its unfair to penalize those that have long mail delays or are
>vacation).  

It's also completely unavoidable.  Live with it.

Eric