drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vicky Markman <vmark...@maprtech.com>
Subject Re: Partition pruning inconsistency
Date Wed, 26 Aug 2015 15:51:00 GMT
There is Jira for (a) as well:
https://issues.apache.org/jira/browse/DRILL-3045

On Wed, Aug 26, 2015 at 8:31 AM, Aman Sinha <asinha@maprtech.com> wrote:

> We have had some issues where the same query run at different times
> (possibly with other queries running concurrently...not sure about the
> concurrency level)  either performed partition pruning or did not.  The
> times where it failed happened due to couple of reasons :
>   (a) allocateNew() in the PruneScanRule failed with an out of memory
> condition
>   (b) the interpreter evaluator encountered an error with a particular
> expression type evaluation
>
> The PruneScanRule currently logs a warning message and does not fail the
> query since this is a performance optimization.  While we will address the
> root cause of (a) and (b) (there's a JIRA open for (b) )  an important
> issue is the inconsistent behavior of a query.
>
> Should we provide a system setting that allows the query to fail in this
> situation ?
> Note that other rules in the optimizer could also fail and some rules  log
> warnings but those failures are very rare, while PruneScan rule is doing
> more complex operations - creating value vectors, doing interpreter
> evaluation - so the chances of something failing increases.
>
> Aman
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message