arrow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From u..@apache.org
Subject arrow git commit: ARROW-1216: [Python] Fix creating numpy array from arrow buffers on python 2
Date Sat, 15 Jul 2017 20:09:21 GMT
Repository: arrow
Updated Branches:
  refs/heads/master d46b7ea3e -> 9ff39f3e9


ARROW-1216: [Python] Fix creating numpy array from arrow buffers on python 2

Author: Philipp Moritz <pcmoritz@gmail.com>

Closes #844 from pcmoritz/numpy-buffer and squashes the following commits:

a891971 [Philipp Moritz] test that base object is set correctly
22bb6b5 [Philipp Moritz] fix creating numpy buffer from arrow buffer on python 2


Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/9ff39f3e
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/9ff39f3e
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/9ff39f3e

Branch: refs/heads/master
Commit: 9ff39f3e9d711bebddbe452ef4f4a3b9b48d5396
Parents: d46b7ea
Author: Philipp Moritz <pcmoritz@gmail.com>
Authored: Sat Jul 15 22:09:16 2017 +0200
Committer: Uwe L. Korn <uwelk@xhochy.com>
Committed: Sat Jul 15 22:09:16 2017 +0200

----------------------------------------------------------------------
 python/pyarrow/io.pxi           | 12 ++++++++++++
 python/pyarrow/tests/test_io.py | 10 ++++++++++
 2 files changed, 22 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/arrow/blob/9ff39f3e/python/pyarrow/io.pxi
----------------------------------------------------------------------
diff --git a/python/pyarrow/io.pxi b/python/pyarrow/io.pxi
index c15be5e..3221185 100644
--- a/python/pyarrow/io.pxi
+++ b/python/pyarrow/io.pxi
@@ -528,6 +528,18 @@ cdef class Buffer:
         buffer.strides = self.strides
         buffer.suboffsets = NULL
 
+    def __getsegcount__(self, Py_ssize_t *len_out):
+        if len_out != NULL:
+            len_out[0] = <Py_ssize_t>self.size
+        return 1
+
+    def __getreadbuffer__(self, Py_ssize_t idx, void **p):
+        if idx != 0:
+            raise SystemError("accessing non-existent buffer segment")
+        if p != NULL:
+            p[0] = <void*> self.buffer.get().data()
+        return self.size
+
 
 cdef shared_ptr[PoolBuffer] allocate_buffer(CMemoryPool* pool):
     cdef shared_ptr[PoolBuffer] result

http://git-wip-us.apache.org/repos/asf/arrow/blob/9ff39f3e/python/pyarrow/tests/test_io.py
----------------------------------------------------------------------
diff --git a/python/pyarrow/tests/test_io.py b/python/pyarrow/tests/test_io.py
index 6258f6d..835f508 100644
--- a/python/pyarrow/tests/test_io.py
+++ b/python/pyarrow/tests/test_io.py
@@ -160,6 +160,16 @@ def test_buffer_bytearray():
     assert result == val
 
 
+def test_buffer_numpy():
+    # Make sure creating a numpy array from an arrow buffer works
+    byte_array = bytearray(20)
+    byte_array[0] = 42
+    buf = pa.frombuffer(byte_array)
+    array = np.frombuffer(buf, dtype="uint8")
+    assert array[0] == byte_array[0]
+    assert array.base == buf
+
+
 def test_buffer_memoryview_is_immutable():
     val = b'some data'
 


Mime
View raw message