arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Lippai (Jira)" <>
Subject [jira] [Created] (ARROW-6774) Reading parquet file is slow
Date Wed, 02 Oct 2019 22:40:00 GMT
Adam Lippai created ARROW-6774:

             Summary: Reading parquet file is slow
                 Key: ARROW-6774
             Project: Apache Arrow
          Issue Type: Improvement
          Components: Rust
    Affects Versions: 0.15.0
            Reporter: Adam Lippai

Using the example at [] is slow.

The snippet 
let reader = SerializedFileReader::new(file).unwrap();
let mut iter = reader.get_row_iter(None).unwrap();
let start = Instant::now();
while let Some(record) = {}
let duration = start.elapsed();
println!("{:?}", duration);
Runs for 17sec for a ~160MB parquet file.

If there is a more effective way to load a parquet file, it would be nice to add it to the

P.S.: My goal is to construct an ndarray from it, I'd be happy for any tips.

This message was sent by Atlassian Jira

View raw message