spamassassin-sysadmins mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Jones <da...@apache.org>
Subject Re: Fwd: Re: Problem with 72_scores.cf generation
Date Wed, 05 Jul 2017 16:15:58 GMT
On 07/04/2017 10:54 PM, Kevin A. McGrail wrote:
> Resending without the screen shot...
> 
> 
> Hi John,
> 
> Thanks.  We spent Friday as well trying to identify what's going on and 
> I worked with Crashplan to restore the data I had for the old solaris 
> box.  It was in an old format because the solaris client was deprecated 
> so it was a lot of juggling.
> 
> In any case, I will A) look at this information again.  and B) I ran 
> crashplan on the ASF servers because ASF Infra doesn't do backups.  I 
> configure it to retain a pretty insane (unlimited) number of revisions 
> as an anti-malware protection.  This means you can go in and restore a 
> specific version of 72_scores.cf

The 72_scores.cf is in SVN so I don't need to restore that file from the 
backups.  What I need to see would have been a temp file created by the 
garescorer compiled executable called 'scores' that would have been in a 
path something like:

trunk-new-rules-set0/masses/gen-set0-5-5.0-6000-ga/scores

I think this was formerly under /tmp if I remember correctly.

Dave

> 
> (removed screenshot, too large and mailing list rejected)
> 
> I put the credentials in SVN for the ASF crashplan and happy to help 
> walk you through things to see if I have a backup of the file you want 
> to compare.
> 
> Regards,
> KAM
> 
> On 7/4/2017 12:59 PM, Dave Jones wrote:
>> Kevin,
>> I have spent about 5 hours this morning trying to track down the 
>> 72_scores.cf generation problem.  I haven't pinpointed the problem yet 
>> but here's what I have found so far:
>>
>> NOTE: su - automc for proper paths below.
>>
>> 1. ~/svn/masses/rule-update-score-gen/generate-new-scores.sh is the 
>> script in question
>>
>> https://svn.apache.org/viewvc/spamassassin/trunk/masses/rule-update-score-gen/generate-new-scores.sh?revision=1798589&view=markup

>>
>>
>> Line 271 runs "runGA"
>>
>> 2. runGA creates 
>> ~/tmp/generate-new-scores/trunk-new-rules-set0/masses/gen-set0-5-5.0-6000-ga/scores

>>
>>
>> This 'scores' file has 345 scores in it.  I wish we had a copy of this 
>> file from mid March to see if it also had around the same number of 
>> scores to confirm the runGA/garescorer is not the problem.  However, 
>> this file is a temp file that used to be in /tmp so it's probably not 
>> backed up anywhere and definitely not in SVN.
>>
>> 3. Back in the generate-new-scores.sh at line 289, the 
>> "extract-new-scores" script creates scores-new from the scores file 
>> but excludes/culls out anything manually scored in 50_scores.cf.
>>
>> The culled scores-new file has the same 42 lines and ends at 
>> MILLION_USD just like our 72_scores.cf so this is the smoking gun but 
>> I haven't found the what pulled the trigger yet.  There is something 
>> different about this step than back on March 15th when we had our last 
>> good 72_scores.cf.
>>
>>
>> THINGS I HAVE CHECKED:
>>
>> At first I thought that 50_scores.cf changed a lot which caused more 
>> exclusion/culling in the 72_scores.cf but that's not it. The revision 
>> only shows a few minor changes in 50_scores.cf:
>>
>> https://svn.apache.org/viewvc/spamassassin/trunk/rules/50_scores.cf?view=log 
>>
>>
>> Next I looked a the garescorer since it get's compiled from 
>> garescorer.c every run.  The garescorer.c is identical in the backups 
>> and in SVN so that's not it.  Again, I wish I had a 'scores' file to 
>> compare to from mid March.
>>
> 


Mime
View raw message