impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (JIRA)" <>
Subject [jira] [Assigned] (IMPALA-4914) TestSpillStress makes flawed assumptions about running concurrently
Date Mon, 20 Mar 2017 18:24:41 GMT


Tim Armstrong reassigned IMPALA-4914:

    Assignee: Tim Armstrong

> TestSpillStress makes flawed assumptions about running concurrently
> -------------------------------------------------------------------
>                 Key: IMPALA-4914
>                 URL:
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Infrastructure
>    Affects Versions: Impala 2.9.0
>            Reporter: Michael Brown
>            Assignee: Tim Armstrong
> I took a look at TestSpillStress and found a bunch of problems with it.
> # It's not being run, because its workload isn't being run exhaustively
> # It can't run, because it doesn't set up a client properly.
> # It looks like it was intended to run in parallel fashion, but custom cluster tests
don't run in parallel.
> The evidence for my claim of #3 is due to the fact that it uses the agg_stress workload,
which says:
> {noformat}
> # This query forces many joins and aggregations with spilling
> # and can expose race conditions in the spilling code if run in parallel
> {noformat}
> 1 and 2 could be fixed very quickly and were fixed at
> 3 is a different thing and requires some thought. We don't have any mechanism to run
custom cluster tests in any sort of parallel way. All custom cluster tests are serial, even
though they lack the serial mark. The reason this is the case is because most custom cluster
tests involve restarting impalad on every *method*. We can't have methods run in parallel
if they will be restarting on top of each other. As such, custom cluster tests are invoked
differently than other e2e tests, and the invocation does not include {{-n}}, which causes
tests to run in parallel.

This message was sent by Atlassian JIRA

View raw message