lucene-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (Jira)" <>
Subject [jira] [Commented] (SOLR-13864) MathExpressionTest non-reproducible failures due to assertions of non-absolutes and randomization beyond test seed
Date Tue, 12 Nov 2019 20:51:00 GMT


ASF subversion and git services commented on SOLR-13864:

Commit b872863da9e06a9f21562830cbd26d24d30a8138 in lucene-solr's branch refs/heads/branch_8x
from Chris M. Hostetter
[;h=b872863 ]

SOLR-13864: SolrTestCaseJ4.getNextAvailablePort() has been deprecated

(cherry picked from commit 603be023feaf3f8e3e739e532b488068710d9097)

> MathExpressionTest non-reproducible failures due to assertions of non-absolutes and randomization
beyond test seed
> ------------------------------------------------------------------------------------------------------------------
>                 Key: SOLR-13864
>                 URL:
>             Project: Solr
>          Issue Type: Test
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Chris M. Hostetter
>            Assignee: Joel Bernstein
>            Priority: Major
>         Attachments: apache_Lucene-Solr-BadApples-Tests-master_531.log.txt
> We're seeing a a fairly steady trickle of MathExpressionTest from various jenkins boxes
going back quite a while ... mostly from testGammaDistribution, but other tests pop up now
and then.
> the crux of the problem with this test seems to break down into 2 categories:
>  # tests that make assumptions about the relative values that will come out of taking
samples from different random distributions that aren't garunteed to be true
>  ** ie: comparing 2 random samples from 2 diff shaped gamma distributions and expecting
one to always be strictly greater then the other. I'm not a stats guy, but my naive understanding
is that on the low end some of these shapes may cross over, so every possible random sample
from one shape is not garunteed to be less then every ossible random sample from a diff shape
>  # the code being tested does it's own randomization outside of the crontrol of the test
framework (or test client)
>  ** this causes the seeds to not reproduce
> ----
> Tests should not be making assertions about random data that aren't 100% garunteed to
be true in all cases (ie: {{random().nextInt(5) < (5.0D + (double) random().nextInt(5))}}
is one thing, {{random().nextInt(5) < (4.99999D + (double) random().nextInt(5))}} is a
diff story.
> Randomized behavior in solr (non-test) code should ideally have some way for being controlled
by the client/tests ... either via a request param used to initialize any new Random instances,
or for example the use of the "tests.seed" property in various places in the code to try and
provide some reproducibility even when the external solr client isn't even aware of randomization
being a factor in the behavior of the code.

This message was sent by Atlassian Jira

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message