drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Re: Heads up on MapR test framework and file ordering (specifically MD-185 tests)
Date Mon, 26 Oct 2015 17:01:21 GMT
My current fix is to remove the specific lists. (A test which is flaky
isn't very valuable.) You can see my current branch at [1]. You can look at
my most recent commit to see the changes I made to the expected results
files to get the tests to report the correct result.

If you want to do file name verification (as opposed to simply numFiles
verification), it seems like you should be doing a structured verification
of the json plan rather than trying to do something with the text plan.

[1] https://github.com/dremio/drill-test-framework

Jacques Nadeau
CTO and Co-Founder, Dremio

On Mon, Oct 26, 2015 at 9:36 AM, Chun Chang <cchang@maprtech.com> wrote:

> Jacques,
> Thanks for the heads up. And what is your proposed fix? I think we can
> easily enhance the test framework to handle file ordering cases. This way,
> we don't need to modify any existing tests.
> Thanks,
> Chun
> On Sat, Oct 24, 2015 at 9:11 PM, Jacques Nadeau <jacques@dremio.com>
> wrote:
> > A large number of tests associated with MD-185 are unintentionally
> brittle.
> > Many of these tests reference more than one file. The tests are planning
> > tests and thus are prone to failure if the file ordering isn't the same
> as
> > what was used for test generation.
> >
> > In the case of these types of tests, people should be cautious of adding
> > expected results that include multiple files in a certain order. I'm
> > working on a patch fix for these specific tests but wanted to let people
> > know to be cautious of these issues in the future.
> >
> > thanks!
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message