arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <wesmck...@gmail.com>
Subject Re: [C++] How can I read streaming parquet file in v0.15.0
Date Thu, 31 Oct 2019 17:46:32 GMT
You will want to use the GetRecordBatchReader C++ API here

https://github.com/apache/arrow/blob/master/cpp/src/parquet/arrow/reader.h#L152

It may not be optimal for your use case. Support for streaming reads
is not yet exposed in Python or other bindings as far as I know.

There is work happening in the C++ Datasets project to better support
this use case.

On Wed, Oct 30, 2019 at 9:28 PM annsshadow <cravenboy@163.com> wrote:
>
>
> hi~
> I hava a question about reading parquet file.
> The offical example is reading the whole file from the local.
> Now I can't get the whole parquet file in the memory, only can fetch it slice by slice
from the network, so how can I use arrow to read the parquet file?
> thank you~

Mime
View raw message