crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Created] (CRUNCH-542) Wider tolerance for flaky scrunch PCollectionTest
Date Tue, 14 Jul 2015 16:12:04 GMT
Josh Wills created CRUNCH-542:

             Summary: Wider tolerance for flaky scrunch PCollectionTest
                 Key: CRUNCH-542
             Project: Crunch
          Issue Type: Improvement
          Components: Scrunch
    Affects Versions: 0.12.0, 0.11.0, 0.10.0
            Reporter: Josh Wills
            Priority: Minor
             Fix For: 0.13.0

One of the Scrunch tests uses an unseeded version of the sample() function that verifies that
it works correctly by ensuring that an actual sampling of elements is within ~ 3 standard
deviations of the expected value. Given this, we expect the test to fail about once every
370 times it is run, or once a year if the tests were run every day.

My issue is that we test about a dozen versions of Crunch automatically in Jenkins every day,
and so I'm having this test fail on at least some version about once every month. I'd like
to bump the control limit up to a little over 5 standard deviations so that the test fails
around once every millennium and/or get rid of the test entirely and only rely on the seeded
versions of the test.

This message was sent by Atlassian JIRA

View raw message