Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 4B845200C5B for ; Thu, 27 Apr 2017 18:11:50 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 47587160BA7; Thu, 27 Apr 2017 16:11:50 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 8F9AF160B98 for ; Thu, 27 Apr 2017 18:11:49 +0200 (CEST) Received: (qmail 59073 invoked by uid 500); 27 Apr 2017 16:11:48 -0000 Mailing-List: contact commits-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@arrow.apache.org Delivered-To: mailing list commits@arrow.apache.org Received: (qmail 59064 invoked by uid 99); 27 Apr 2017 16:11:48 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Apr 2017 16:11:48 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 64C89DFC4A; Thu, 27 Apr 2017 16:11:48 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: uwe@apache.org To: commits@arrow.apache.org Message-Id: <18f1a38607ca46bd90cb223bcc600359@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: arrow git commit: ARROW-900: [Python] Fix UnboundLocalError in ParquetDatasetPiece.read Date: Thu, 27 Apr 2017 16:11:48 +0000 (UTC) archived-at: Thu, 27 Apr 2017 16:11:50 -0000 Repository: arrow Updated Branches: refs/heads/master 81be9c667 -> 03dce9dca ARROW-900: [Python] Fix UnboundLocalError in ParquetDatasetPiece.read Author: Wes McKinney Closes #607 from wesm/ARROW-900 and squashes the following commits: 81f8394 [Wes McKinney] Fix UnboundLocalError in ParquetDatasetPiece.read Project: http://git-wip-us.apache.org/repos/asf/arrow/repo Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/03dce9dc Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/03dce9dc Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/03dce9dc Branch: refs/heads/master Commit: 03dce9dcab1df587f2293decf49708f872aaad3d Parents: 81be9c6 Author: Wes McKinney Authored: Thu Apr 27 18:11:44 2017 +0200 Committer: Uwe L. Korn Committed: Thu Apr 27 18:11:44 2017 +0200 ---------------------------------------------------------------------- python/pyarrow/parquet.py | 3 +++ python/pyarrow/tests/test_parquet.py | 14 ++++++++++++++ 2 files changed, 17 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/arrow/blob/03dce9dc/python/pyarrow/parquet.py ---------------------------------------------------------------------- diff --git a/python/pyarrow/parquet.py b/python/pyarrow/parquet.py index 94ad227..21359f1 100644 --- a/python/pyarrow/parquet.py +++ b/python/pyarrow/parquet.py @@ -208,6 +208,9 @@ class ParquetDatasetPiece(object): reader = self._open(open_file_func) elif file is not None: reader = ParquetFile(file) + else: + # try to read the local path + reader = ParquetFile(self.path) if self.row_group is not None: table = reader.read_row_group(self.row_group, columns=columns, http://git-wip-us.apache.org/repos/asf/arrow/blob/03dce9dc/python/pyarrow/tests/test_parquet.py ---------------------------------------------------------------------- diff --git a/python/pyarrow/tests/test_parquet.py b/python/pyarrow/tests/test_parquet.py index 8c446af..bb3a9ed 100644 --- a/python/pyarrow/tests/test_parquet.py +++ b/python/pyarrow/tests/test_parquet.py @@ -493,6 +493,20 @@ def test_read_single_row_group(): @parquet +def test_parquet_piece_read(tmpdir): + df = _test_dataframe(1000) + table = pa.Table.from_pandas(df) + + path = tmpdir.join('parquet_piece_read.parquet').strpath + pq.write_table(table, path, version='2.0') + + piece1 = pq.ParquetDatasetPiece(path) + + result = piece1.read() + assert result.equals(table) + + +@parquet def test_parquet_piece_basics(): path = '/baz.parq'