arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bhavitvya Malik <bhavitvya.ma...@gmail.com>
Subject pyarrow.lib.ArrowInvalid: Can only convert 1-dimensional array values
Date Thu, 11 Feb 2021 10:33:25 GMT
Hi,
It's a follow up question for #9462
<https://github.com/apache/arrow/issues/9462>. Rewriting the issue here:

I came to know that pyarrow has this limitation of not storing
> N-dimensional array. After looking into this issue, I decided to represent
> a N-dimensional array as a list of arrays i.e.
> data = np.zeros((5, 3), dtype=np.uint8)
> data = list(data)
> inorder to preserve the dtype but when it comes to typecasting and writing
> it into array (from list) pyarrow.array(data, type=type) it gives the
> following error:
> pyarrow.lib.ArrowInvalid: Could not convert [0 0 0] with type
> numpy.ndarray: tried to convert to int
> Is there any way to avoid this issue? I just want to preserve the dtype
> from numpy array before converting it to list so that while writing it to
> pyarrow array format I can recognise its dtype and subsequently write it in
> that numpy dtype format.


I tried it with a 3D numpy array and it gave me this error even though it's
working fine with 2D numpy arrays. Can you please look into this?

pyarrow.lib.ArrowInvalid: Can only convert 1-dimensional array values


My current pyarrow version is 2.0.0 and i tried it with pyarrow==3.0.0 too


Thanks,
Bhavitvya

Mime
View raw message