impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Huaisi Xu (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-2.2.0 5.4.x) CDH-41243: Parquet scanner regression on wide tables (part 2)
Date Thu, 23 Jun 2016 22:32:13 GMT
Huaisi Xu has submitted this change and it was merged.

Change subject: CDH-41243: Parquet scanner regression on wide tables (part 2)
......................................................................


CDH-41243: Parquet scanner regression on wide tables (part 2)

IMPALA-2473 introduced a check that prevent row batches growing
beyond 8MB, but it has a corner case that when an empty row
batch is larger than 8MB, it returns this row batch immediately
after it materialize one row, essentailly setting batch_size=1.

Revert "IMPALA-2473: reduce scanner memory usage"

This reverts commit cecb4cf4c5bfe4d21afc2f650880e5bdda14b024.

Change-Id: Id21c26771cd9f5239da4e07a6c59c5126b4d8a0b
Reviewed-on: http://gerrit.cloudera.org:8080/3417
Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
Tested-by: Huaisi Xu <hxu@cloudera.com>
---
M be/src/exec/data-source-scan-node.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/hdfs-table-sink.cc
M be/src/exec/hdfs-table-sink.h
M be/src/runtime/row-batch.h
M testdata/workloads/functional-query/queries/QueryTest/scanners.test
7 files changed, 26 insertions(+), 64 deletions(-)

Approvals:
  Huaisi Xu: Verified
  Tim Armstrong: Looks good to me, approved



-- 
To view, visit http://gerrit.cloudera.org:8080/3417
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Id21c26771cd9f5239da4e07a6c59c5126b4d8a0b
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.2.0_5.4.x
Gerrit-Owner: Huaisi Xu <hxu@cloudera.com>
Gerrit-Reviewer: Huaisi Xu <hxu@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message