impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Behm (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3845: Split up hdfs-parquet-scanner.cc into more files/components.
Date Thu, 14 Jul 2016 16:30:26 GMT
Hello Marcel Kornacker,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/3596

to look at the new patch set (#5).

Change subject: IMPALA-3845: Split up hdfs-parquet-scanner.cc into more files/components.
......................................................................

IMPALA-3845: Split up hdfs-parquet-scanner.cc into more files/components.

This patch refactors hdfs-parquet-scanner.cc into several files.
The new responsibilities of each file/component are roughly as follows:

hdfs-parquet-scanner.cc
- Creates column readers aud uses them to materializes row batches.
- Evaluates runtime filters and conjuncts, populates row batch queue.

parquet-metadata-utils.h/cc
- Contains utilities for validating Parquet file metadata.
- Parses the schema of a Parquet file into our internal schema
  representation.
- Resolves SchemaPaths (e.g. from a table descriptor) against
  the internal representation of the Parquet file schema.

parquet-column-readers.h
- Contains the per-column data reading, parsing and value
  materialization logic.

Testing: A private core/hdfs run passed.

Change-Id: I4c5fd46f9c1a0ff2a4c30ea5a712fbae17c68f92
---
M be/src/exec/CMakeLists.txt
M be/src/exec/base-sequence-scanner.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-parquet-scanner.h
M be/src/exec/hdfs-rcfile-scanner.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/hdfs-scanner.h
M be/src/exec/hdfs-text-scanner.cc
A be/src/exec/parquet-column-readers.cc
A be/src/exec/parquet-column-readers.h
A be/src/exec/parquet-metadata-utils.cc
A be/src/exec/parquet-metadata-utils.h
A be/src/exec/parquet-scratch-tuple-batch.h
M be/src/exprs/expr-value.h
M be/src/runtime/runtime-state.cc
M be/src/runtime/runtime-state.h
M be/src/util/debug-util.cc
M be/src/util/debug-util.h
18 files changed, 2,683 insertions(+), 2,462 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/96/3596/5
-- 
To view, visit http://gerrit.cloudera.org:8080/3596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4c5fd46f9c1a0ff2a4c30ea5a712fbae17c68f92
Gerrit-PatchSet: 5
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message