Return-Path: X-Original-To: apmail-parquet-commits-archive@minotaur.apache.org Delivered-To: apmail-parquet-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 78F0218FEB for ; Wed, 23 Mar 2016 01:41:32 +0000 (UTC) Received: (qmail 18987 invoked by uid 500); 23 Mar 2016 01:41:32 -0000 Delivered-To: apmail-parquet-commits-archive@parquet.apache.org Received: (qmail 18953 invoked by uid 500); 23 Mar 2016 01:41:32 -0000 Mailing-List: contact commits-help@parquet.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@parquet.apache.org Delivered-To: mailing list commits@parquet.apache.org Received: (qmail 18942 invoked by uid 99); 23 Mar 2016 01:41:32 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Mar 2016 01:41:32 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 21A54E0044; Wed, 23 Mar 2016 01:41:32 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: wesm@apache.org To: commits@parquet.apache.org Message-Id: <8143115ce717415880fd84b2ba38352a@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: parquet-cpp git commit: PARQUET-559: Enable external RandomAccessSource as input to the ParquetFileReader Date: Wed, 23 Mar 2016 01:41:32 +0000 (UTC) Repository: parquet-cpp Updated Branches: refs/heads/master 486aa105e -> fbb25bf3e PARQUET-559: Enable external RandomAccessSource as input to the ParquetFileReader Author: Deepak Majeti Closes #83 from majetideepak/PARQUET-559 and squashes the following commits: d7f9c47 [Deepak Majeti] modified ParquetFileReader API b5269f4 [Deepak Majeti] modified travis ci script to print log on failure 940de12 [Deepak Majeti] fixed memory leak 6de0b70 [Deepak Majeti] print logs e5ea341 [Deepak Majeti] print failure 9462954 [Deepak Majeti] try fixing test failure 3a0a1d7 [Deepak Majeti] modified External Stream 9d8b44c [Deepak Majeti] fixed formatting 11b3e6f [Deepak Majeti] Enable an external InputStream as a source to the ParquetFileReader Project: http://git-wip-us.apache.org/repos/asf/parquet-cpp/repo Commit: http://git-wip-us.apache.org/repos/asf/parquet-cpp/commit/fbb25bf3 Tree: http://git-wip-us.apache.org/repos/asf/parquet-cpp/tree/fbb25bf3 Diff: http://git-wip-us.apache.org/repos/asf/parquet-cpp/diff/fbb25bf3 Branch: refs/heads/master Commit: fbb25bf3ed27f149c3b59a182f8f47e4148fb860 Parents: 486aa10 Author: Deepak Majeti Authored: Tue Mar 22 18:41:24 2016 -0700 Committer: Wes McKinney Committed: Tue Mar 22 18:41:24 2016 -0700 ---------------------------------------------------------------------- .travis.yml | 2 +- src/parquet/file/reader.cc | 17 +++++++++++------ src/parquet/file/reader.h | 5 +++++ 3 files changed, 17 insertions(+), 7 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/fbb25bf3/.travis.yml ---------------------------------------------------------------------- diff --git a/.travis.yml b/.travis.yml index 8d49b50..a540380 100644 --- a/.travis.yml +++ b/.travis.yml @@ -31,7 +31,7 @@ matrix: script: - make lint - make -j4 || exit 1 - - ctest || exit 1 + - ctest || { cat $TRAVIS_BUILD_DIR/parquet-build/Testing/Temporary/LastTest.log; exit 1; } - sudo pip install cpp_coveralls - export PARQUET_ROOT=$TRAVIS_BUILD_DIR - $TRAVIS_BUILD_DIR/ci/upload_coverage.sh http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/fbb25bf3/src/parquet/file/reader.cc ---------------------------------------------------------------------- diff --git a/src/parquet/file/reader.cc b/src/parquet/file/reader.cc index ff61941..91016a6 100644 --- a/src/parquet/file/reader.cc +++ b/src/parquet/file/reader.cc @@ -74,6 +74,16 @@ ParquetFileReader::~ParquetFileReader() { Close(); } +std::unique_ptr ParquetFileReader::Open( + std::unique_ptr source, MemoryAllocator* allocator) { + auto contents = SerializedFile::Open(std::move(source), allocator); + + std::unique_ptr result(new ParquetFileReader()); + result->Open(std::move(contents)); + + return result; +} + std::unique_ptr ParquetFileReader::OpenFile(const std::string& path, bool memory_map, MemoryAllocator* allocator) { std::unique_ptr file; @@ -84,12 +94,7 @@ std::unique_ptr ParquetFileReader::OpenFile(const std::string } file->Open(path); - auto contents = SerializedFile::Open(std::move(file), allocator); - - std::unique_ptr result(new ParquetFileReader()); - result->Open(std::move(contents)); - - return result; + return Open(std::move(file), allocator); } void ParquetFileReader::Open(std::unique_ptr contents) { http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/fbb25bf3/src/parquet/file/reader.h ---------------------------------------------------------------------- diff --git a/src/parquet/file/reader.h b/src/parquet/file/reader.h index 3a54cfb..9daae53 100644 --- a/src/parquet/file/reader.h +++ b/src/parquet/file/reader.h @@ -30,6 +30,7 @@ namespace parquet_cpp { class ColumnReader; +class RandomAccessSource; struct RowGroupStatistics { int64_t num_values; @@ -101,6 +102,10 @@ class ParquetFileReader { static std::unique_ptr OpenFile(const std::string& path, bool memory_map = true, MemoryAllocator* allocator = default_allocator()); + static std::unique_ptr Open( + std::unique_ptr source, + MemoryAllocator* allocator = default_allocator()); + void Open(std::unique_ptr contents); void Close();