arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <wesmck...@gmail.com>
Subject Re: [C++]How to use BufferedInputStream in parquet::arrow::OpenFile()?
Date Sun, 24 Jan 2021 17:42:41 GMT
Note that the buffered stream option in parquet::ReaderProperties is
what you want here

https://github.com/apache/arrow/blob/master/cpp/src/parquet/properties.h#L74

On Fri, Jan 22, 2021 at 10:34 PM Micah Kornfield <emkornfield@gmail.com> wrote:
>
> InputStream and RandomAccessFile are inherently different types that support different
operations.  RandomAccessFile can emulate and InputStream but not vice-versa.  Parquet files
have a footer that requires reading first, so an InputStream cannot be used.
>
> On Wed, Dec 16, 2020 at 6:30 PM annsshadow <cravenboy@163.com> wrote:
>>
>> Hi~all
>>
>> I try to use BufferedInputStream to reduce the overhead of some small read from the
network.
>>
>> The pseudo codes are below:
>>
>>
>> ```
>>
>> //get the buffered input stream
>>
>> auto buffered_result = arrow::io::BufferedInputStream::Create()
>>
>> _buffered_infile = buffered_result.ValueOrDie();
>>
>>
>> // follow the example codes, I want to open a parquet file like that
>>
>> // but it meets compiler error: could not convert from 'std::shared_ptr<arrow::io::BufferedInputStream>'
to 'std::shared_ptr<arrow::io::RandomAccessFile>'
>>
>> PARQUET_THROW_NOT_OK(parquet::arrow::OpenFile(_buffered_infile, arrow::default_memory_pool(),
&_reader_parquet));
>>
>>
>> //the declaration of OpenFile
>>
>> //Status OpenFile(std::shared_ptr<::arrow::io::RandomAccessFile> file, MemoryPool*
pool,
>>
>>                 std::unique_ptr<FileReader>* reader)
>>
>> ```
>>
>>
>> How can I use it correctly?
>>
>> Thanks all~
>>
>>

Mime
View raw message