arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <>
Subject Re: pyarrow.lib.ArrowInvalid: Can only convert 1-dimensional array values
Date Thu, 11 Feb 2021 15:48:15 GMT
Can you provide more detail about what you are trying? You've showed
some exception here but haven't showed the exact code that results in
those exceptions

On Thu, Feb 11, 2021 at 4:34 AM Bhavitvya Malik
<> wrote:
> Hi,
> It's a follow up question for #9462. Rewriting the issue here:
>> I came to know that pyarrow has this limitation of not storing N-dimensional array.
After looking into this issue, I decided to represent a N-dimensional array as a list of arrays
>> data = np.zeros((5, 3), dtype=np.uint8)
>> data = list(data)
>> inorder to preserve the dtype but when it comes to typecasting and writing it into
array (from list) pyarrow.array(data, type=type) it gives the following error:
>> pyarrow.lib.ArrowInvalid: Could not convert [0 0 0] with type numpy.ndarray: tried
to convert to int
>> Is there any way to avoid this issue? I just want to preserve the dtype from numpy
array before converting it to list so that while writing it to pyarrow array format I can
recognise its dtype and subsequently write it in that numpy dtype format.
> I tried it with a 3D numpy array and it gave me this error even though it's working fine
with 2D numpy arrays. Can you please look into this?
> pyarrow.lib.ArrowInvalid: Can only convert 1-dimensional array values
> My current pyarrow version is 2.0.0 and i tried it with pyarrow==3.0.0 too
> Thanks,
> Bhavitvya

View raw message