impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Brown (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5281: stress test: introduce stricter pass guidelines
Date Fri, 30 Jun 2017 16:52:41 GMT
Michael Brown has posted comments on this change.

Change subject: IMPALA-5281: stress test: introduce stricter pass guidelines
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/7282/2/tests/stress/concurrent_select.py
File tests/stress/concurrent_select.py:

PS2, Line 443: self._num_queries_timedout.value - self._num_queries_cancelled.value
> just curious, what's the relationship between "timed out" and "cancelled" q
> I always thought timed out means hung and cancelled were the deliberately cancelled queries
(to test cancellation)

In terms of the final report distinction, you're essentially correct. The short answer is
that cancelled queries are also considered "timed out" as far as the stress test execution
is concerned.

Long answer:

At the query execution level, all queries are given some sort of expected time duration in
which to complete. If they don't, they are cancelled, and a timed_out Boolean is set. This
control flow is the same for all queries. You can see it here:

https://github.com/apache/incubator-impala/blob/master/tests/stress/concurrent_select.py#L821

At a level above, that duration is set based on whether or not the query is going to be intentionally
cancelled.

https://github.com/apache/incubator-impala/blob/master/tests/stress/concurrent_select.py#L639

This gives an intentionally-cancelling query time to "ramp up" before being cancelled.

Once a query finished, if it has timed out, a flag is examined to see whether the query was
meant to be intentionally cancelled. If yes, great. If not, there's logic below to handle
some of the cases.

There is other logic that increments the report:

https://github.com/apache/incubator-impala/blob/master/tests/stress/concurrent_select.py#L729

I suppose we could change the flow here not to need the arithmetic.


-- 
To view, visit http://gerrit.cloudera.org:8080/7282
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I9f2174a527193ae01be45b8ed56315c465883346
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Brown <mikeb@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: David Knupp <dknupp@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Matthew Mulder <mmulder@cloudera.com>
Gerrit-Reviewer: Michael Brown <mikeb@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message