impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Behm (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5282: Handle overflows in computeResourceProfile().
Date Mon, 05 Jun 2017 21:40:48 GMT
Alex Behm has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/7084

Change subject: IMPALA-5282: Handle overflows in computeResourceProfile().
......................................................................

IMPALA-5282: Handle overflows in computeResourceProfile().

There were two bugs:
1. An overflow in computeResourceProfile() could lead to
   negative resource estimates which later lead to a failed
   Preconditions check.
2. Failure to compute the resource profile failed the query.

- I went through the computeResourceProfile() implementation
  of all data sinks and plan nodes and add code for handling
  overflows.
- Exceptions in computeResourceProfile() are logged as a
  warning, but the query continues to run.

Testing:
- Reproduced the issue locally with the DDL provided in the
  JIRA. I manually set huge NDV stats to trigger overflow.

Change-Id: I8a83a76141853d3274f812e5a531f456e1b110b1
---
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
3 files changed, 13 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/84/7084/1
-- 
To view, visit http://gerrit.cloudera.org:8080/7084
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8a83a76141853d3274f812e5a531f456e1b110b1
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <alex.behm@cloudera.com>

Mime
View raw message