Return-Path: X-Original-To: apmail-crunch-dev-archive@www.apache.org Delivered-To: apmail-crunch-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 14B3918306 for ; Thu, 16 Jul 2015 15:00:18 +0000 (UTC) Received: (qmail 52949 invoked by uid 500); 16 Jul 2015 15:00:05 -0000 Delivered-To: apmail-crunch-dev-archive@crunch.apache.org Received: (qmail 52915 invoked by uid 500); 16 Jul 2015 15:00:05 -0000 Mailing-List: contact dev-help@crunch.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@crunch.apache.org Delivered-To: mailing list dev@crunch.apache.org Received: (qmail 52897 invoked by uid 500); 16 Jul 2015 15:00:05 -0000 Delivered-To: apmail-incubator-crunch-dev@incubator.apache.org Received: (qmail 52894 invoked by uid 99); 16 Jul 2015 15:00:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Jul 2015 15:00:05 +0000 Date: Thu, 16 Jul 2015 15:00:05 +0000 (UTC) From: "Josh Wills (JIRA)" To: crunch-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (CRUNCH-542) Wider tolerance for flaky scrunch PCollectionTest MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CRUNCH-542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Wills updated CRUNCH-542: ------------------------------ Attachment: CRUNCH-542b.patch Alright, I'm cool to drop it entirely-- we already have the seeded version in place. Thanks! > Wider tolerance for flaky scrunch PCollectionTest > ------------------------------------------------- > > Key: CRUNCH-542 > URL: https://issues.apache.org/jira/browse/CRUNCH-542 > Project: Crunch > Issue Type: Improvement > Components: Scrunch > Affects Versions: 0.10.0, 0.11.0, 0.12.0 > Reporter: Josh Wills > Priority: Minor > Fix For: 0.13.0 > > Attachments: CRUNCH-542.patch, CRUNCH-542b.patch > > > One of the Scrunch tests uses an unseeded version of the sample() function that verifies that it works correctly by ensuring that an actual sampling of elements is within ~ 3 standard deviations of the expected value. Given this, we expect the test to fail about once every 370 times it is run, or once a year if the tests were run every day. > My issue is that we test about a dozen versions of Crunch automatically in Jenkins every day, and so I'm having this test fail on at least some version about once every month. I'd like to bump the control limit up to a little over 5 standard deviations so that the test fails around once every millennium and/or get rid of the test entirely and only rely on the seeded versions of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)