Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 5B168200CC3 for ; Sat, 15 Jul 2017 22:09:24 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 596221668CD; Sat, 15 Jul 2017 20:09:24 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (unknown [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 9ED951668C4 for ; Sat, 15 Jul 2017 22:09:23 +0200 (CEST) Received: (qmail 46689 invoked by uid 500); 15 Jul 2017 20:09:22 -0000 Mailing-List: contact commits-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@arrow.apache.org Delivered-To: mailing list commits@arrow.apache.org Received: (qmail 46680 invoked by uid 99); 15 Jul 2017 20:09:22 -0000 Received: from Unknown (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 15 Jul 2017 20:09:22 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id EF803EEE19; Sat, 15 Jul 2017 20:09:21 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: uwe@apache.org To: commits@arrow.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: arrow git commit: ARROW-1216: [Python] Fix creating numpy array from arrow buffers on python 2 Date: Sat, 15 Jul 2017 20:09:21 +0000 (UTC) archived-at: Sat, 15 Jul 2017 20:09:24 -0000 Repository: arrow Updated Branches: refs/heads/master d46b7ea3e -> 9ff39f3e9 ARROW-1216: [Python] Fix creating numpy array from arrow buffers on python 2 Author: Philipp Moritz Closes #844 from pcmoritz/numpy-buffer and squashes the following commits: a891971 [Philipp Moritz] test that base object is set correctly 22bb6b5 [Philipp Moritz] fix creating numpy buffer from arrow buffer on python 2 Project: http://git-wip-us.apache.org/repos/asf/arrow/repo Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/9ff39f3e Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/9ff39f3e Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/9ff39f3e Branch: refs/heads/master Commit: 9ff39f3e9d711bebddbe452ef4f4a3b9b48d5396 Parents: d46b7ea Author: Philipp Moritz Authored: Sat Jul 15 22:09:16 2017 +0200 Committer: Uwe L. Korn Committed: Sat Jul 15 22:09:16 2017 +0200 ---------------------------------------------------------------------- python/pyarrow/io.pxi | 12 ++++++++++++ python/pyarrow/tests/test_io.py | 10 ++++++++++ 2 files changed, 22 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/arrow/blob/9ff39f3e/python/pyarrow/io.pxi ---------------------------------------------------------------------- diff --git a/python/pyarrow/io.pxi b/python/pyarrow/io.pxi index c15be5e..3221185 100644 --- a/python/pyarrow/io.pxi +++ b/python/pyarrow/io.pxi @@ -528,6 +528,18 @@ cdef class Buffer: buffer.strides = self.strides buffer.suboffsets = NULL + def __getsegcount__(self, Py_ssize_t *len_out): + if len_out != NULL: + len_out[0] = self.size + return 1 + + def __getreadbuffer__(self, Py_ssize_t idx, void **p): + if idx != 0: + raise SystemError("accessing non-existent buffer segment") + if p != NULL: + p[0] = self.buffer.get().data() + return self.size + cdef shared_ptr[PoolBuffer] allocate_buffer(CMemoryPool* pool): cdef shared_ptr[PoolBuffer] result http://git-wip-us.apache.org/repos/asf/arrow/blob/9ff39f3e/python/pyarrow/tests/test_io.py ---------------------------------------------------------------------- diff --git a/python/pyarrow/tests/test_io.py b/python/pyarrow/tests/test_io.py index 6258f6d..835f508 100644 --- a/python/pyarrow/tests/test_io.py +++ b/python/pyarrow/tests/test_io.py @@ -160,6 +160,16 @@ def test_buffer_bytearray(): assert result == val +def test_buffer_numpy(): + # Make sure creating a numpy array from an arrow buffer works + byte_array = bytearray(20) + byte_array[0] = 42 + buf = pa.frombuffer(byte_array) + array = np.frombuffer(buf, dtype="uint8") + assert array[0] == byte_array[0] + assert array.base == buf + + def test_buffer_memoryview_is_immutable(): val = b'some data'