spamassassin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Spamassassin Wiki] Update of "SaUpdateBackend" by JustinMason
Date Mon, 23 Jan 2006 08:24:40 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Spamassassin Wiki" for change notification.

The following page has been changed by JustinMason:
http://wiki.apache.org/spamassassin/SaUpdateBackend

The comment on the change is:
document sa-update backend

New page:
= Rule Promotion and the sa-update Backend =

'sa-update' updates are created on the zone, nightly, by a cron job.
Here are some details as to how this works.

At 0830 UTC, the '''build/mkupdates/run_nightly''' script is run from svn
trunk.  This runs the script '''build/mkupdates/listpromotable''', which
queries the rule-QA web app for the previous day's rule accuracy figures, and
determines which rules from the rules source dirs (rulesrc/core and
rulesrc/sandbox/*) are to be ''promoted'' to the active ruleset.

== Promotion Criteria ==

You can see the previous day's accuracy figures
[http://buildbot.spamassassin.org/ruleqa/?daterev=last-night at this URL].
Promotable rules show up in dark text on the RuleQaApp, non-promoted-by-default
rules show up in light grey text.

The promotion criteria are detailed on some other wiki page,
RulesProjPromotion perhaps? (TODO).

Anyway, using those figures, and some other criteria:

 * "tflags nopublish" rules are never published
 * rules with a {{{T_}}} prefix to their names are never published
 * "tflags publish", "net", "userconf" or "learn" rules are always published
 * rules that fail lint -- if that can be attributed to the rule! -- are not published
 * rules that require a plugin that '''build/mkupdates/listpromotable''' thinks is not part
of the default plugin set, are not published

A proposed update checkin is created with a new list of 'active' rules,
in the file 'rules/active.list'; this is immediately checked in.

== The Active List ==

The active list, in the form of the file '''rules/active.list''' is checked in to SVN trunk,
and is used by '''build/mkrules''' to determine which rules are active.

 * Rules marked active, and their dependencies (if they're meta rules), are written to '''rules/72_active.cf''';

 * Rules which are not in the active list, but which were loaded from a sandbox directory
in ''rulesrc/sandbox/*'', are written to '''rules/70_sandbox.cf''', and renamed to always
include a {{{T_}}} prefix;

 * Rules which are not in the active list, but which were loaded from a core-rules dir in
''rulesrc/core'', are written to '''rules/70_inactive.cf'''.

build/mkrules attempts to keep the contents of those files consistent.  In
other words, meta subrules for a rule in the active list will always likewise
be copied to the active file -- even if they are sandbox T_ rules.  It
will also ignore rules that are inside an {{{ifplugin}}} block for
a plugin that is not available.

'''rules/70_sandbox.cf''' and '''rules/70_inactive.cf''' are used for
development, and for mass-checks, but will not be installed during
{{{make install}}} and are not packaged in sa-update tarballs.
Rules in '''rules/70_sandbox.cf''' are the only ones used in the "bbmass'' preflight mass-checks.

== The sa-update Tarball ==

'''build/mkrules''' is run, using the active list to create
'''rules/72_active.cf''', '''rules/70_sandbox.cf''' and
'''rules/70_inactive.cf'''.   The latter two files are then
discarded, and {{{spamassassin --lint}}} run to ensure the
ruleset is still in a consistent state.   If lint fails,
the update process is abandoned.

A {{{make install}}} is run, to a temporary directory, and the resulting
ruleset is packaged using tar, linted again from that dir, and then signed and
checksummed.

The DNS zone is updated using the SVN revision number, and the
tarball copied into place.

=== Administrivia ===

The cron jobs run as the user ''updatesd''.   The crontab looks like:

{{{
30 8 * * * bash /home/updatesd/svn/spamassassin/build/mkupdates/run_nightly > /var/www/buildbot.spamassassin.org/updatesd/mkupdates.txt
2>&1
50 8 * * * bash /home/updatesd/svn/spamassassin/build/mkupdates/run_part2 > /var/www/buildbot.spamassassin.org/updatesd/mkupdatespt2.txt
2>&1
}}}

Mime
View raw message