drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Altekruse (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (DRILL-4259) Add new functional tests to ensure that failures can be detected independent of the testing environment
Date Thu, 14 Jan 2016 21:41:39 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094201#comment-15094201
] 

Jason Altekruse edited comment on DRILL-4259 at 1/14/16 9:41 PM:
-----------------------------------------------------------------

That's what I was thinking, although I'm not very familiar with the setup of the regression
tests. I know that some tests set options themselves by just including the alter sessions
statements in the test files. I also know that we added an enhancement for the framework to
set options globally, so that we could leave the decimal tests turned on [1] (even though
we turned of the feature by default due to some lingering pref and correctness issues).

I think what I would be looking to do would be configure a few different combinations of configurations
that could be applied over an entire suite, so that we can run a number of tests with a few
different configurations. Doing this by manually changing the test files to include near copies
with the new options set doesn't seem like the right way to go. Let me know if this seems
like a reasonable approach. I can work on the enhancement to the framework to enable it if
necessary.

[1] - https://github.com/mapr/drill-test-framework/blob/master/framework/before-run.sql


was (Author: jaltekruse):
That's what I was thinking, although I'm not very familiar with the setup of the regression
tests. I know that some tests set options themselves by just including the alter sessions
statements in the test files. I also know that we added an enhancement for the framework to
set options globally, so that we could leave the decimal tests turned on [1] (even though
we turned of the feature by default due to some lingering pref and correctness issues).

I think what I would be looking to do would be configure a few different combinations of configurations
that could be applied over an entire suite, so that we can run a number of tests with a few
different configurations. Doing this by manually changing the test files to include near copes
with the new options set doesn't seem like the right way to go. Let me know if this seems
like a reasonable approach. I can work on the enhancement to the framework to enable it if
necessary.

[1] - https://github.com/mapr/drill-test-framework/blob/master/framework/before-run.sql

> Add new functional tests to ensure that failures can be detected independent of the testing
environment
> -------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-4259
>                 URL: https://issues.apache.org/jira/browse/DRILL-4259
>             Project: Apache Drill
>          Issue Type: Test
>            Reporter: Jason Altekruse
>
> In DRILL-4243 an out of memory issue was fixed after a change to the memory allocator
made memory limits more strict. While the regression tests had been run by the team at Dremio
prior to merging the patch, running the tests on a cluster with more cores changed the memory
limits on the queries and caused several tests to fail.
> While changes of this magnitude are not going to be common, we should have a test suite
that reliably fails independent of the environment it is run (assuming that there are sufficient
resources for the tests to run).
> It would be good to at least try to reproduce this failure on a few different setups
(cores, nodes in cluster) by adjusting available configuration options and adding tests with
those different configurations so that the tests will fail in different environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message