arrow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From w...@apache.org
Subject arrow git commit: ARROW-1446: [Python] Add (very slow) large memory unit test for int32 overflow in PARQUET-1090
Date Wed, 06 Sep 2017 23:27:28 GMT
Repository: arrow
Updated Branches:
  refs/heads/master 3d285c69d -> d9f895ebd


ARROW-1446: [Python] Add (very slow) large memory unit test for int32 overflow in PARQUET-1090

Bug fixed in https://github.com/apache/parquet-cpp/pull/389

Author: Wes McKinney <wes.mckinney@twosigma.com>

Closes #1055 from wesm/ARROW-1446 and squashes the following commits:

97c634c9 [Wes McKinney] Add (very slow) large memory unit test for PARQUET-1090


Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/d9f895eb
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/d9f895eb
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/d9f895eb

Branch: refs/heads/master
Commit: d9f895ebde94a21bd2bf91085a186683fb521e48
Parents: 3d285c6
Author: Wes McKinney <wes.mckinney@twosigma.com>
Authored: Wed Sep 6 19:27:23 2017 -0400
Committer: Wes McKinney <wes.mckinney@twosigma.com>
Committed: Wed Sep 6 19:27:23 2017 -0400

----------------------------------------------------------------------
 python/pyarrow/tests/test_parquet.py | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/arrow/blob/d9f895eb/python/pyarrow/tests/test_parquet.py
----------------------------------------------------------------------
diff --git a/python/pyarrow/tests/test_parquet.py b/python/pyarrow/tests/test_parquet.py
index fa9455b..ae5c28f 100644
--- a/python/pyarrow/tests/test_parquet.py
+++ b/python/pyarrow/tests/test_parquet.py
@@ -1239,3 +1239,17 @@ def test_write_to_dataset_with_partitions(tmpdir):
 @parquet
 def test_write_to_dataset_no_partitions(tmpdir):
     _test_write_to_dataset_no_partitions(str(tmpdir))
+
+
+@pytest.mark.large_memory
+@parquet
+def test_large_table_int32_overflow():
+    size = np.iinfo('int32').max + 1
+
+    arr = np.ones(size, dtype='uint8')
+
+    parr = pa.Array.from_pandas(arr, type=pa.uint8())
+
+    table = pa.Table.from_arrays([parr], names=['one'])
+    f = io.BytesIO()
+    _write_table(table, f)


Mime
View raw message