arrow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From u..@apache.org
Subject arrow git commit: ARROW-900: [Python] Fix UnboundLocalError in ParquetDatasetPiece.read
Date Thu, 27 Apr 2017 16:11:48 GMT
Repository: arrow
Updated Branches:
  refs/heads/master 81be9c667 -> 03dce9dca


ARROW-900: [Python] Fix UnboundLocalError in ParquetDatasetPiece.read

Author: Wes McKinney <wes.mckinney@twosigma.com>

Closes #607 from wesm/ARROW-900 and squashes the following commits:

81f8394 [Wes McKinney] Fix UnboundLocalError in ParquetDatasetPiece.read


Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/03dce9dc
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/03dce9dc
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/03dce9dc

Branch: refs/heads/master
Commit: 03dce9dcab1df587f2293decf49708f872aaad3d
Parents: 81be9c6
Author: Wes McKinney <wes.mckinney@twosigma.com>
Authored: Thu Apr 27 18:11:44 2017 +0200
Committer: Uwe L. Korn <uwelk@xhochy.com>
Committed: Thu Apr 27 18:11:44 2017 +0200

----------------------------------------------------------------------
 python/pyarrow/parquet.py            |  3 +++
 python/pyarrow/tests/test_parquet.py | 14 ++++++++++++++
 2 files changed, 17 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/arrow/blob/03dce9dc/python/pyarrow/parquet.py
----------------------------------------------------------------------
diff --git a/python/pyarrow/parquet.py b/python/pyarrow/parquet.py
index 94ad227..21359f1 100644
--- a/python/pyarrow/parquet.py
+++ b/python/pyarrow/parquet.py
@@ -208,6 +208,9 @@ class ParquetDatasetPiece(object):
             reader = self._open(open_file_func)
         elif file is not None:
             reader = ParquetFile(file)
+        else:
+            # try to read the local path
+            reader = ParquetFile(self.path)
 
         if self.row_group is not None:
             table = reader.read_row_group(self.row_group, columns=columns,

http://git-wip-us.apache.org/repos/asf/arrow/blob/03dce9dc/python/pyarrow/tests/test_parquet.py
----------------------------------------------------------------------
diff --git a/python/pyarrow/tests/test_parquet.py b/python/pyarrow/tests/test_parquet.py
index 8c446af..bb3a9ed 100644
--- a/python/pyarrow/tests/test_parquet.py
+++ b/python/pyarrow/tests/test_parquet.py
@@ -493,6 +493,20 @@ def test_read_single_row_group():
 
 
 @parquet
+def test_parquet_piece_read(tmpdir):
+    df = _test_dataframe(1000)
+    table = pa.Table.from_pandas(df)
+
+    path = tmpdir.join('parquet_piece_read.parquet').strpath
+    pq.write_table(table, path, version='2.0')
+
+    piece1 = pq.ParquetDatasetPiece(path)
+
+    result = piece1.read()
+    assert result.equals(table)
+
+
+@parquet
 def test_parquet_piece_basics():
     path = '/baz.parq'
 


Mime
View raw message