Mailing-List: contact users-help@spamassassin.apache.org; run by ezmlm
Precedence: bulk
Received-SPF: pass (nike.apache.org: domain of dfs@roaringpenguin.com
 designates 70.38.112.54 as permitted sender)
Date: Tue, 21 Jun 2011 10:23:31 -0400
From: "David F. Skoll" <dfs@roaringpenguin.com>
To: users@spamassassin.apache.org
Subject: Re: High Performance Bayes Database Configuration?
Message-ID: <20110621102331.2e59f1a1@hydrogen.roaringpenguin.com>
In-Reply-To: <4E00A553.5000708@junkemailfilter.com>
References: <4E00A553.5000708@junkemailfilter.com>
Organization: Roaring Penguin Software Inc.
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Tue, 21 Jun 2011 07:06:11 -0700
Marc Perkel <support@junkemailfilter.com> wrote:

> Trying to get MySQL bays working in a high volume environment.
> Dedicated MySQL server with SSD drives. Can someone send me a sample
> my.cnf file and make other suggestings to keep it running wihout
> database corruption and other MySQL "features"? Or - should I be
> using some other DB?

We've tried various ways of storing Bayes data (we have our own Bayes
implementation, so this discussion may not correspond exactly with the
SA implementation.)  After trying Berkeley DB files and PostgreSQL---we
would never use MySQL for any data we care about---we finally settled
on Dan Bernstein's CDB format.  It has by far the best performance.
See: http://www.dmo.ca/blog/benchmarking-hash-databases-on-large-data/
Take a look at the "Random Reads" timings.  CDB is 6 times faster than
Berkeley DB!

CDB is read-only, which means when you want to do Bayes training, you
have to rewrite the entire database.  This is not an issue for our
system because of how we do Bayes training, but it may be an issue
with the standard sa-learn.

Regards,

David.