arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <wesmck...@gmail.com>
Subject Re: [python] not an arrow file
Date Sun, 24 Jan 2021 17:41:08 GMT
Can you show your C++ code?

On Sun, Jan 24, 2021 at 8:10 AM Teh, Kenneth M. <teh@anl.gov> wrote:

> Just started with arrow...
>
> I wrote a record batch to a file using ipc::MakeFileWriter to create a
> writer and writer->WriteRecordBatch in a C++ program and tried to read it
> in python with:
>
> [] import pyarrow as pa
> [] reader = pa.ipc.open_file("myfile")
>
>
> It raises the ArrowInvalid with the message "not an arrow file".
>
> If I write it out as a Table in feather format, I can read it in python.
> But I want to write large files on the order of 100GB or more and then read
> them back into python as pandas dataframes or something similar.
>
> So, I switched to using an ipc writer.
>
> Can something point me in the right direction?  Thanks.
>
> Ken
>

Mime
View raw message