arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fernando Herrera <fernando.j.herr...@gmail.com>
Subject [RUST] Reading parquet
Date Sat, 23 Jan 2021 12:33:41 GMT
Hi all,

A quick question regarding reading a parquet file. What is the best way to
read a parquet file and keep it in memory to do data analysis?

What I'm doing now is using the record reader from the
ParquetFileArrowReader and then I read all the record batches from the
file. I keep the batches in memory in a vector of record batches. This way
I have access to them to do some aggregations I need from the file.

Is there another way to do this?

Thanks,
Fernando

Mime
View raw message