spamassassin-sysadmins mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Jones <da...@apache.org>
Subject Re: Potential problem with do-stable-update-with-scores cron job
Date Fri, 02 Jun 2017 13:10:29 GMT
On 06/02/2017 07:40 AM, Kevin A. McGrail wrote:
> I think you are right.  And that the system needs a run to create the 
> files for the scores on the new server.

I ran a manual "force" with 9 masscheck contributors last night so we 
have a test file out there that I have asked the ruleqa list for help 
with QA'ing the update file before I enable DNS updates for sa-update:

wget http://bbmass.spamassassin.org/updates/1797164.tar.gz
wget http://bbmass.spamassassin.org/updates/1797164.tar.gz.sha1
wget http://bbmass.spamassassin.org/updates/1797164.tar.gz.asc

sa-update -D -v --install=1797164.tar.gz

> 
> On 5/30/2017 8:25 PM, Kevin A. McGrail wrote:
>> It's quite possible bb does update rule qa.  Don't take what I say as 
>> absolute...  Will look at this more tomorrow.
>> Regards,
>> KAM
>>

On the ruleqa list yesterday, John Hardin confirmed that buildbot was 
running before to centrally process the full uploaded corpora.  Sigh. 
This is going to be difficult to figure out and get going again so we 
need to focus on the distributed masscheck contributions first which is 
close to being good.

The http://ruleqa.spamassassin.org site is not going to be updating 
until we figure out what was running that.

Dave

>> On May 30, 2017 8:15:16 PM EDT, Dave Jones <davej@apache.org> wrote:
>>> If you look on the sa-vm1 box in /etc/cron.d/automc you will see all of
>>>
>>> the cron jobs that I found from the /home/kmcgrail/SAcron mbox.  I
>>> updated the bottom of the InfraNote2017 wiki page with what these cron
>>> jobs do.  I think this is the core of the ruleqa process that allows us
>>>
>>> to validate rules from the submitted corpus and start sa-update working
>>>
>>> again but the http://ruleqa.spamassassin.org site is not getting
>>> updated.  This is my next thing to tackle.  Now that I know buildbot
>>> wasn't involved that will same me a lot of time.  I am concerned that
>>> where ever this part was running wasn't backed up. I may have to take
>>> your advice and post on the dev mailing list to see if anyone knows
>>> anything or has some backups somewhere.
>>>
>>> Dave
>>>
>>> On 05/30/2017 07:00 PM, Kevin A. McGrail wrote:
>>>> Ahh. This was for the quick masschecks for the preflight rules. I
>>>> don't think it is used nor is the corpora it is run against
>>> maintained.
>>>> Is there parts of rule qa that aren't working other than these quick
>>>> checks?
>>>> Regards,
>>>> KAM
>>>>
>>>> On May 30, 2017 7:50:40 PM EDT, Dave Jones <davej@apache.org> wrote:
>>>>
>>>>      In looking into what updates the RuleQAApp web interface for
>>>>      http://ruleqa.spamassassin.org, I found this:
>>>>
>>>>      https://wiki.apache.org/spamassassin/PreflightBuildBot
>>>>
>>>>      Something has to do some processing to update the data folder
>>> that the
>>>>      ruleqa.cgi uses for the http://ruleqa.spamassassin site.  If we
>>> can
>>>>      simply run the commands/steps from a cron'd script, that would be
>>> better
>>>>      but I am not finding any such script so far.  May have to create
>>> one.
>>>>      Buildbot configs are in SVN:
>>>>
>>> https://svn.apache.org/repos/asf/spamassassin/trunk/build/buildbot/
>>>>      Dave
>>>>
>>>>      On 05/30/2017 05:54 PM, Kevin A. McGrail wrote:
>>>>
>>>>          Well, what are we using buildbot for? I'm trying to remember
>>>>          and it might not be needed. On 5/30/2017 5:08 PM, Dave Jones
>>>>          wrote:
>>>>
>>>>              I think I have figured out the primary hurdle I was
>>>>              hitting when I wrote that last email on 5/27. I found
>>> some
>>>>              stuff in the backups the very long and hard way to get a
>>>>              little closer. Now I am at the buildbot setup and
>>>>              discovery stage. I installed buildbot on sa-vm1 but there
>>>>              is a huge change in versions so the old master config
>>> file
>>>>              in SVN has to be converted into the new version which
>>> will
>>>>              take me some time as I have to learn buildbot from
>>>>              scratch. Dave On 05/30/2017 12:17 PM, Kevin A. McGrail
>>> wrote:
>>>>                  On 5/27/2017 3:12 PM, Dave Jones wrote:
>>>>
>>>>                      This script:
>>> https://svn.apache.org/repos/asf/spamassassin/trunk/masses/rule-update-score-gen/do-nightly-rescore-example

>>>
>>>>                      refers to
>>>>                      '/home/dos/sa-score-gen/nightly-rescore-via-cron'
>>>>                      which is not in any backups under
>>>>                      /usr/local/spamassassin/backups. A few lines down
>>>>                      in the script it refers to 'generate-new-scores'
>>>>                      which exists in:
>>> https://svn.apache.org/repos/asf/spamassassin/trunk/masses/rule-update-score-gen/

>>>
>>>>                      It appears that his is a different
>>>>                      'generate-new-scores' than what might have been
>>> in
>>>>                      the missing /home/dos/sa-score-gen since it's
>>>>                      doing some rsync'ing to pull in corpus which
>>>>                      should have been on the same server? Where the
>>>>                      servers separated before? Was
>>>>                      rsync.spamassassin.org
>>>>                      <http://rsync.spamassassin.org> on a different
>>> box
>>>>                      than where these cron jobs ran?
>>>>
>>>>                  Dave, What machines ran what, splitting and
>>>>                  recombining servers, etc. is too much for me to
>>>>                  remember, sorry. I can try and dig too but it's
>>>>                  confusing to me as well. However, an important thing
>>>>                  is are we running the correct scripts. Since I don't
>>>>                  see /home/dos on spamassassin-vm1 in my backups, I'm
>>>>                  guessing it was on a different server. Regards, KAM
>>>>
> 


Mime
View raw message