From users-return-116893-archive-asf-public=cust-asf.ponee.io@spamassassin.apache.org Thu Jan 4 17:40:22 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 467CA180657 for ; Thu, 4 Jan 2018 17:40:22 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 3666E160C2B; Thu, 4 Jan 2018 16:40:22 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2D288160C28 for ; Thu, 4 Jan 2018 17:40:21 +0100 (CET) Received: (qmail 22164 invoked by uid 500); 4 Jan 2018 16:40:19 -0000 Mailing-List: contact users-help@spamassassin.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list users@spamassassin.apache.org Received: (qmail 22153 invoked by uid 99); 4 Jan 2018 16:40:19 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Jan 2018 16:40:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id F0FEB180893 for ; Thu, 4 Jan 2018 16:40:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.302 X-Spam-Level: X-Spam-Status: No, score=-0.302 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, KAM_NUMSUBJECT=0.5, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=ena.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id trIH6f1-1B-j for ; Thu, 4 Jan 2018 16:40:17 +0000 (UTC) Received: from smtp5i.ena.net (smtp5i.ena.net [96.5.1.13]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 4CCF85F1B9 for ; Thu, 4 Jan 2018 16:40:17 +0000 (UTC) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03lp0050.outbound.protection.outlook.com [216.32.180.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by smtp5i.ena.net (Postfix) with ESMTPS id DC7001480A64; Thu, 4 Jan 2018 10:40:03 -0600 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ena.com; s=default; t=1515084004; bh=8AsL2p716aVpxIcfSSp9Hr0EwyXEBwyXL7YxWugNWdY=; h=Subject:To:References:From:Date:In-Reply-To; b=YnuEw8IIg6zHkO1+oAJ3A+En3qCHNcSHFh3tXRkj+7nA3Nx+OGy2msPx237ohTlun 13ukyK3Wg5YyFULsrIvQA6zWNKpvQV/ZPmZt+z3fDRsvD49JahoCIkb6QikB0P3Ma2 mwoY437bkrcxHrMaZA6eiT7//rnQDv7/vkqZcUxw= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=djones@ena.com; Received: from djones5.jonesol.com (76.187.196.227) by CY4PR0201MB3492.namprd02.prod.outlook.com (2603:10b6:910:95::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.386.5; Thu, 4 Jan 2018 16:40:01 +0000 Subject: Re: Question about BAYES_999 To: users@spamassassin.apache.org References: <94c97587-bab9-09d3-6495-f5a10d63e3f9@ena.com> <2682b16e-3533-d39f-d956-e3563a80d684@thelounge.net> <20180102135707.60a5683e@gumby.homeunix.com> <7851885D-AF24-4F89-BFC3-48D4484F57D6@kreme.com> <20180104104615.GB10990@fantomas.sk> <70b0bb47-6244-e6a1-da90-eac735efaf8b@ena.com> <20180104160402.3160ffd6@gumby.homeunix.com> From: David Jones Message-ID: Date: Thu, 4 Jan 2018 10:40:49 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <20180104160402.3160ffd6@gumby.homeunix.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [76.187.196.227] X-ClientProxiedBy: BN6PR08CA0081.namprd08.prod.outlook.com (2603:10b6:404:b6::19) To CY4PR0201MB3492.namprd02.prod.outlook.com (2603:10b6:910:95::10) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ab2dd28c-11fe-40be-2e11-08d55391cd20 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(5600026)(4604075)(2017052603307)(7153060);SRVR:CY4PR0201MB3492; X-Microsoft-Exchange-Diagnostics: 1;CY4PR0201MB3492;3:ki5p779mIJe+aYIgzA01kSbIwBA+YDD3zfpLUhjZQs0kA4/UBRiCWaB2w0nxVhcGsFf8sOza4SobStRg+2kGi/4iki7G6mN8xdoPNgUeNv5tIR25+voqAmQYvvE1NHv+b9TynFr6stphnRYYsoNwKzBAu0Ve7BLC8KQyBpAzjhEroHJwfDkKLeMg48riJicPH/VFF9Z9stiOxuQCiy6L91RrhIGvLLll9dz1E9wZdf46KpAGu1yLbNXZ5XwSA3Rt;25:OkHIgVLj79P4qxwmy6no5k1RwOqO4ZFIh2up3DBQBvL3e8hZ3sG5RYoNyHd5Dh5ALpPVMk4ouYZfETtNnUW6OdJyuuEkUCL0qhZ+6srV2onpAWGRWoaJzOOcnJXrr4ddajuxGAEQAj+oqVvDPo7j+rNcgkKC1OVbjoJ10CHpDi8QGq7v2atFtlBQYW0JoSy05YyUJwNWHGEaf8P0YKMd8YP3iVE73W3qZGpK9GtK7nlTGoQ79eZ7GPlGiB53c/RejyX5MUdvvX8FDLpgzQWqT9AQuloKPu6mwVyMHCiWK5pj11GNoLacvTlblktj7MNXENA5fZ20hjHHXtv5dNhLhQ==;31:ULXau5VEVfFuWHfSYUWv5xoas9PC5g30DgjtZdt+ovGPsmceUcZVWXNgerpstnYHSI+oRQ22C74joXyEfDOxS0l0vCOrfgMGS9pKt/rdwPqLsN3y/4vtxFIkGzQgtZ0Kwj6SQdFbajgEYxoxdktaD8YQK27HucUnexhKcrE0ViIlk6NpwEwEdLEeJSYZLYvVvf+dQ4hhLAVPosVmpX5WOFWxlaQfy1yWs73FOVax634= X-MS-TrafficTypeDiagnostic: CY4PR0201MB3492:|CY4PR0201MB3492: X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(228869891846254)(211171220733660); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040470)(2401047)(8121501046)(5005006)(93006095)(93001095)(3231023)(944501075)(3002001)(10201501046)(6041268)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(20161123560045)(20161123564045)(6072148)(201708071742011);SRVR:CY4PR0201MB3492;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:CY4PR0201MB3492; X-Microsoft-Exchange-Diagnostics: 1;CY4PR0201MB3492;4:7x5GnqD697gciORpW4HueI0EtTcLhpmJsx1KidusVEiL9Hb5wZGK4hghiKehNz3GdEkROmpskzCCjqNLBtXNU288Lki+60xkvhwwxpbeKDKiznWzozxVly+NI0NGJPB0LBqXeILx08OTpZL9eKcbkHc1IRzc2xGFnQWZMeIkCIqiN3X6sDSLOJaW1dUOQnYNgBCIVSClRi+5y3sHyAqfoAnq/C+xPU6FOwxurNNHhRIn68UausnMgwMrmGow5Lp/Ms2F7CJSVfw5Ph5AbTlXb7IceDL1o20l3/oCrJ9hukgQYSvBxmWmRaThk/8iIdzXrNHMUdB7xhFHnxR3LqJnyel/cyw1quoZk1vTwKZgwqE= X-Forefront-PRVS: 054231DC40 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(346002)(39380400002)(396003)(366004)(39860400002)(376002)(199004)(24454002)(189003)(76104003)(316002)(6916009)(36756003)(31696002)(478600001)(2950100002)(6666003)(2870700001)(69596002)(83506002)(6486002)(53936002)(229853002)(5660300001)(105586002)(66066001)(65956001)(65806001)(7116003)(93886005)(6512007)(58126008)(65826007)(16526018)(86362001)(25786009)(47776003)(68736007)(2351001)(67846002)(2361001)(97736004)(31686004)(53546011)(2906002)(386003)(106356001)(6506007)(52146003)(23676004)(76176011)(6246003)(50466002)(2486003)(52116002)(7736002)(8936002)(8676002)(6116002)(81156014)(81166006)(64126003)(305945005)(53416004)(3846002);DIR:OUT;SFP:1101;SCL:1;SRVR:CY4PR0201MB3492;H:djones5.jonesol.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; Received-SPF: None (protection.outlook.com: ena.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTRQUjAyMDFNQjM0OTI7MjM6RW4wUENFakV4SENpanBUYlpWQ2JOWnY1?= =?utf-8?B?VXloeWhJdWN6VW51RGp1RzdUbms3VGxDYUtOSU1uS2cxczVtUjRJejZYUEpD?= =?utf-8?B?T3FGQUcyTFU3U3BEYmd5WGdRNTdqYzNoYXhnZUQvUW5RdFZHeUQ5UmVYNFZt?= =?utf-8?B?ZC9lTVhQV205cVFzYnNLVXRiWkdDTnZhcDExa3FvdVZzQXpnRCsxMUhWQWsr?= =?utf-8?B?NTJidTkxc3ZOZUdoQ1VrbEJqd2FaZ1prT01oSitHbGtWdTNlR0ZGSkFhVS9W?= =?utf-8?B?dExkWTVGTTZOSHNZS0lIdytEYVFTRG43dWVpQ3hnYkRVeVl0VnRLRVRSalJR?= =?utf-8?B?Z0dQRm15Y2hJSXRZaWtGeHczVkVVTWRTQ1NwRHcrWFNTbG1Cd2RmZVpMd3M3?= =?utf-8?B?QVoxd3NKdlBGNnJlMlV5R0g4ZWJVbERHTEFQek1kZ1FPWEVSRm82VStoeGw1?= =?utf-8?B?ZHlROSs5aisrLzBDZTNQMTBvUHoxQjdqMFY2T0ZHNzZsSFpHRFBpbG80U2dD?= =?utf-8?B?TmFRdnFrWldMMnNyWS9IUlFvd0NvbWhMZEhiZ3N3ZHJxbEo4WGRYOWhEaVV1?= =?utf-8?B?REdUNll5WEVkZ3p0bjhMT2hXdlpoSHBqdWUvYjh0OXBBb3JEWjNMY2gwRHpz?= =?utf-8?B?c3dVN1dGT2dKRm1zQktPMklHRmpZNFFqOStjRnNtWnl6Z3NKTUYzeExtWUUv?= =?utf-8?B?QlQrMEZPcnI5OEg3d0ZlY3lRWm1MYlIvWFl4aFkrZmJNczdwSFlxZE5waS9O?= =?utf-8?B?Ky9NbGR2RWVxRjc1RU0vZnB5RlF2Qks2STkvOG4renB3TEFHRTNYRlZiR1cz?= =?utf-8?B?ZDNmMThGWUVkbU9YYWRUb0dUQmVFMTRxTkVwdHgxczJ5VmFNcDV1eGl4M25W?= =?utf-8?B?bjJnU2diQ3NqVVZLLy90UjRZYnRwcE8wTUNQNXliMzlGRWYrOFZEWHR2dkdC?= =?utf-8?B?V2hKT1FST2NaZ3NZd0VVeC9neGVnckZMV3BDbjNDMVpURWUweVFHRTg0KzNE?= =?utf-8?B?VzNIWDQ0SXVZRzhpOWU5UStIZ2crbld3THJ6SG1KZVRoMU1MZStIYWJySGZX?= =?utf-8?B?YW9MR1Y5WXFxVG52YXV6WTloT2o4aEwxa25qcHZBcU5LTDZxTGpLZnIrOGRI?= =?utf-8?B?ZjloN0hBazhFTVQvcGlVMUF1ZWkwL2tYRE96QjN5VGZMc1RPYUVJcjAyWVho?= =?utf-8?B?My9nbWFZZVpLbFB6dkw0NkI2MlJVVXBFc0VvRDhHeXdQajJSOFRQcEtFbjdV?= =?utf-8?B?SGJXLzFWYjhNV09oUUFDcmVBZUY1SzBydnU5YXBFWWRHWTVxL2NBeGphSk8w?= =?utf-8?B?b3djVU5ZM3pVVjY5b3djU2lpVUZOR0hibVFIdXAzUkxHWStFUndIQ1pQbVRn?= =?utf-8?B?aGFKczlQYld5WWEvejIwR1VEeWhUVnBBeW1yUDJ4cm4xcktodWEwTCtzQitz?= =?utf-8?B?Z3Y5alFiMjQ1NDc4aHdqYWNrRDAvM1R3aEhIWWswWmJmYzUyMC9UZHh5Skk4?= =?utf-8?B?SzNPZ3MxcDRQNEZyem9JcTAyY1hsQ0JYekVib3pPZVFaeHVwOWdWZHBIVXhR?= =?utf-8?B?eFFnZVIrRlg3ZjJFM2NTU25jZWtLRXFoSkZ6RndlT3BLMWUwbXl4NktFU1g0?= =?utf-8?B?akFia2U5SXpMQUplQ3FBQnBlQ0I0UVZkT1BYMjFOcEdaUnA2ZFoxZDkxcG95?= =?utf-8?B?Y2I0NUpXU0RsdDg4WnZrNUxWR2JBWHNjd0pFRU81TmdPc1Vab05HVGVCck9N?= =?utf-8?B?M1FCMHFQMHRsTUppSTFFajFiMDdoSzBibWtSdXpRUVkyK205bFRlRFA2QkVP?= =?utf-8?B?ckNsVHBXV0xEN2JBeFFmQlA2T1lIdHFCdXZQMUxOc25hdXFMZHRPakdCOEpq?= =?utf-8?B?UHlaQW5kWFgwUmtGeWlaTGhIbDA0NGcrR1drMWhJbzNiTVFMZzVVdzVQN2ZM?= =?utf-8?B?OURpZmZlS1pidkh6MW5IVXJnN0xTL0ROakpQSWtsY3lGRmY5ZUQ4S2dHLzlQ?= =?utf-8?B?UGgwQUozQVlyTlVWRWxZYjlDWm9Yc3lmWVgyTldnPT0=?= X-Microsoft-Exchange-Diagnostics: 1;CY4PR0201MB3492;6:wcTG8pBrGAGlo+kKXxfXGNohKsWVddw4JDYqWIiBTPBIFTdBv8/srlmVW3y3TWyrzyULVkHYjWO8c3fUWUsTPVya+X7I+XfSblBeqyfofb1neEY7B1m4vhc/S/7zkZoj66/9mY2Dd75A0wsGY2mQKagrCc4DVphWbe5Blxr/dsbtZQYH3137RDFkyAr46gtmN8ot1b5QEJEwO0YAWWRiFAXAV1RByUnf+pMoj63f+Xi/Ex+T+jJZhGmNtpjUdaflgRBB5AfhyAFa+q2A/6JmCnH5B86WdUmBD2epUtYzYST2NcCLly5p2anKP116VNKSxMB1VV5dO/avQWeXLNBEcsmvDJ4xZmv0iifW/6mZIk8=;5:dL+P1Vwvc6EseoNDHjUwXktrcBEeearYjv9fWXF9XMsUoAGlDAGQYmZKitmxjvZYFGUBJFIHXr6L/FMNGSRQhRdvud4UzhhJmagqdW2iMnOjcgryaX4afApOZ5CMpXBt7ZVANjIyjYtTcZXAoQGOJ/6P9Uw2KUfoJFgpxWijBSQ=;24:1C3u/Hb58mBcfjX00mXJAOXzCvUR862lJH+7K4e6bL1YUrj4WBw80ZBBhDNFrVj7ppg5PXcZvwOmIUkx5rzftc3IdvBT9qzMeNoXXm4Nt88=;7:ivsQ3A1XehA9FT6Vg80i1gIb4dO2botkAfgwkP3c5jdvDR6OmxsNmxdjptQSd4FGgoh6rX6kSA8vxb4lEmJTsAAQdiQYArXyiN8+sgGBz0FZHbXPD448Xi2VgkD6GeBsnMR59sNTf1e+JZgcZGa01E2CiKlfp9bt2+eydkuxPULjQex6v5+kP3ByPXuZf3cRSN0AHRZpWs16CCsLbQe62aIc/n1REycJI+QIKK+6vv87Vb1OwO1ycShfCa3kf6B/ SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: ena.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jan 2018 16:40:01.6313 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ab2dd28c-11fe-40be-2e11-08d55391cd20 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 6dc38cd4-4d4f-4826-9649-17854289d170 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR0201MB3492 X-ENA-MailScanner-Information: Report abuse to abuse@ena.com and include the next header value X-ENA-MailScanner-ID: DC7001480A64.A0532 X-ENA-MailScanner: No viruses found X-ENA-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=2.398, required 4, BAYES_00 -2.20, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10, ENA_FREEMAIL_HOSTKARMA_OFFSET 2.50, ENA_FREEMAIL_MSPIKE_H2_OFFSET 1.20, ENA_FREEMAIL_SENDERSCORE_90_100_OFFSET 2.20, ENA_FROM_O365_PROXY 4.20, ENA_MSGID_ENA_COM 0.20, KAM_NUMSUBJECT 0.50, RCVD_IN_DNSWL_NONE -0.20, RCVD_IN_HOSTKARMA_W -2.50, RCVD_IN_MSPIKE_H2 -1.20, RCVD_IN_SENDERSCORE_90_100 -2.20, SPF_HELO_PASS -0.00, SPF_PASS -0.00) X-ENA-MailScanner-SpamScore: ss X-ENA-MailScanner-From: djones@ena.com X-ENA-MailScanner-Watermark: 1515688804.69415@zeq/xgualvsTSJCgCCWJKA On 01/04/2018 10:04 AM, RW wrote: > On Thu, 4 Jan 2018 08:02:48 -0600 > David Jones wrote: > >> On 01/04/2018 04:46 AM, Matus UHLAR - fantomas wrote: >>>> On 2 Jan 2018, at 07:17, David Jones djones@ena.com> wrote: >>>>> I haven't redefined these rules from what I can tell by searching >>>>> my local rules.  I would think that if I had done this, then >>>>> there would be consistent non-hits of BAYES_99 with BAYES_999 all >>>>> of the time.  This is only happening a small percentage of the >>>>> time. >>> >>> On 02.01.18 15:39, @lbutlr wrote: >>>> Checking my mail I see an incidence rate of this of about 0.5%, >>>> which matches the rate you posted earlier. >>> >>> amavis? >>> >> >> I am seeing this problem on my MailScanner filters as well: >> >> # grep BAYES_999 maillog-20171231 | wc -l >> 9172 >> # grep BAYES_999 maillog-20171231 | grep -v "BAYES_99 " | wc -l >> 4 > > Are you sure that's right? It's a radically different frequency from > 0.5% and 0.8%. IIWY I'd look at the 4 and check they are what you think > they are and not something like > My production MailScanner instance has a highly tuned MTA in front of it so SA doesn't see as much spam. The amavis instance is intentionally open to more spam to collect for the nightly masscheck processing. > ... rules: meta test FOO has dependency 'BAYES_999' with a zero score > If I had BAYES_99 set to a zero score, it would never show up in my logs. > I get some bogus warnings like this. > > You need something to make sure it's a result line and some boundary > checks like \bBAYES_99\b might help too. > MailScanner log output looks like this: Dec 31 07:30:46 smtp2i.ena.net MailScanner[26137]: Message 8902A148068E.ACC23 from 106.10.241.143 (novak5450@att.net) to k12tn.net is spam, SpamAssassin (not cached, score=35.679, required 4, autolearn=spam, BAYES_99 5.20, BAYES_999 0.20, DCC_CHECK 2.20, DIGEST_MULTIPLE 0.29, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10, DMARC_NONE 0.01, ... Pretty sure my grep'ing above is good. > > I can't reproduce the problem using spamc/spamd from 3.4.1 with perl > 5.24.3 on FreeBSD 11.1 with Berkeley DB. > > I don't have any missing BAYES_99 hits in my corpus headers and to > check it's not a recent bug I rescanned ~5k spams and checked the logs. > I appreciate you checking to help us determine how widespread this issue is and possibly narrow it down. Here is my SA setup that is common to both my amavis and MailScanner: SA 3.4.1 Shared Bayes DB in Redis Share the same local configs for custom rules Since we have at least one other person on this list reporting the same problem, I don't think my local custom rules are the problem. -- David Jones