drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Altekruse <altekruseja...@gmail.com>
Subject Re: [jira] [Created] (DRILL-4259) Add new functional tests to ensure that failures can be detected independent of the testing environment
Date Tue, 12 Jan 2016 16:02:32 GMT
Just realized replying to these messages does not put comments on the JIRA,
do you want to copy your comment there to consolidate the discussion on
this topic?

On Tue, Jan 12, 2016 at 8:01 AM, Jason Altekruse <altekrusejason@gmail.com>
wrote:

> That's what I was thinking, although I'm not very familiar with the setup
> of the regression tests. I know that some tests set options themselves by
> just including the alter sessions statements in the test files. I also know
> that we added an enhancement for the framework to set options globally, so
> that we could leave the decimal tests turned on [1] (even though we turned
> of the feature by default due to some lingering pref and correctness
> issues).
>
> I think what I would be looking to do would be configure a few different
> combinations of configurations that could be applied over an entire suite,
> so that we can run a number of tests with a few different configurations.
> Doing this by manually changing the test files to include near copes with
> the new options set doesn't seem like the right way to go. Let me know if
> this seems like a reasonable approach. I can work on the enhancement to the
> framework to enable it if necessary.
>
> [1] -
> https://github.com/mapr/drill-test-framework/blob/master/framework/before-run.sql
>
>
> On Mon, Jan 11, 2016 at 12:15 PM, Chun Chang <cchang@maprtech.com> wrote:
>
>> Can we come up with a list of different configuration option settings we
>> want to run? The automation framework supports running with different
>> system settings now.
>>
>> On Mon, Jan 11, 2016 at 9:43 AM, Jason Altekruse (JIRA) <jira@apache.org>
>> wrote:
>>
>> > Jason Altekruse created DRILL-4259:
>> > --------------------------------------
>> >
>> >              Summary: Add new functional tests to ensure that failures
>> can
>> > be detected independent of the testing environment
>> >                  Key: DRILL-4259
>> >                  URL: https://issues.apache.org/jira/browse/DRILL-4259
>> >              Project: Apache Drill
>> >           Issue Type: Test
>> >             Reporter: Jason Altekruse
>> >
>> >
>> > In DRILL-4243 an out of memory issue was fixed after a change to the
>> > memory allocator made memory limits more strict. While the regression
>> tests
>> > had been run by the team at Dremio prior to merging the patch, running
>> the
>> > tests on a cluster with more cores changed the memory limits on the
>> queries
>> > and caused several tests to fail.
>> >
>> > While changes of this magnitude are not going to be common, we should
>> have
>> > a test suite that reliably fails independent of the environment it is
>> run
>> > (assuming that there are sufficient resources for the tests to run).
>> >
>> > It would be good to at least try to reproduce this failure on a few
>> > different setups (cores, nodes in cluster) by adjusting available
>> > configuration options and adding tests with those different
>> configurations
>> > so that the tests will fail in different environments.
>> >
>> >
>> >
>> > --
>> > This message was sent by Atlassian JIRA
>> > (v6.3.4#6332)
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message