impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4539: fix bug when scratch batch references I/O buffers
Date Wed, 07 Dec 2016 22:47:30 GMT
Tim Armstrong has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/5406

Change subject: IMPALA-4539: fix bug when scratch batch references I/O buffers
......................................................................

IMPALA-4539: fix bug when scratch batch references I/O buffers

The bug occurs when scanning uncompressed plain-encoded Parquet
string data.

Testing:
I could manually reproduce it reliably with ASAN and
--disable_mem_pools=true using the following steps:

  # The repro is more predictable when there is no row batch queue.
  set mt_dop=1;
  # A unique string column should switch to plain encoding
  set compression_codec=none;
  create table big_uuids stored as parquet as select uuid() from tpch_20_parquet.lineitem;
  # The repro requires that some rows are filtered out, so that we end
  # up in a state where the output batch is full before all rows are
  # copied from the scratch batch
  select ndv(_c0) from big_uuids where substring(_c0, 1, 2) != 'a1' limit 10;

After the fix it no longer reproduces.

I do not yet have a practical test case that triggers the bug on a
normal ASAN setup. I will continue to try to create one.

Change-Id: Ic27e7251e0f633cb694b506f6eb62beed6e66ad9
---
M be/src/exec/hdfs-parquet-scanner.cc
1 file changed, 8 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/5406/1
-- 
To view, visit http://gerrit.cloudera.org:8080/5406
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic27e7251e0f633cb694b506f6eb62beed6e66ad9
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message