impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Juan Yu (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-2.5.0_5.7.0) IMPALA-1886/IMPALA-2154: Add support for multi-stream bz2/gzip compressed files.
Date Fri, 26 Feb 2016 05:57:24 GMT
Hello Internal Jenkins, Skye Wanderman-Milne, Dan Hecht,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/2219

to look at the new patch set (#16).

Change subject: IMPALA-1886/IMPALA-2154: Add support for multi-stream bz2/gzip compressed
files.
......................................................................

IMPALA-1886/IMPALA-2154: Add support for multi-stream bz2/gzip compressed files.

Fix a bug in which Impala only reads the first stream
of a multi-stream bz2/gzip file.
Changes the bz2 decoder to read the file in a streaming
fashion rather than reading the entire file into memory
before it can be decompressed.

Change-Id: Icbe617d03a69953f0bf3aa0f7c30d34bc612f9f8
(cherry picked from commit b6d0b4e059329633dc50f1f73ebe35b7ac317a8e)
---
M be/src/exec/hdfs-text-scanner.cc
M be/src/exec/hdfs-text-scanner.h
M be/src/util/codec.cc
M be/src/util/codec.h
M be/src/util/decompress-test.cc
M be/src/util/decompress.cc
M be/src/util/decompress.h
M common/thrift/generate_error_codes.py
M testdata/data/README
A testdata/data/data-bzip2.bz2
A testdata/data/data-pbzip2.bz2
A testdata/data/large_bzip2.bz2
A testdata/data/large_pbzip2.bz2
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M testdata/workloads/functional-query/queries/DataErrorsTest/hdfs-scan-node-errors.test
A testdata/workloads/functional-query/queries/QueryTest/text-bzip-scan.test
M tests/query_test/test_compressed_formats.py
18 files changed, 524 insertions(+), 186 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/19/2219/16
-- 
To view, visit http://gerrit.cloudera.org:8080/2219
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Icbe617d03a69953f0bf3aa0f7c30d34bc612f9f8
Gerrit-PatchSet: 16
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.5.0_5.7.0
Gerrit-Owner: Juan Yu <jyu@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Juan Yu <jyu@cloudera.com>
Gerrit-Reviewer: Skye Wanderman-Milne <skye@cloudera.com>

Mime
View raw message