arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luke <virtuall...@gmail.com>
Subject parquet file in S3, is there a way to read a subset of all the columns in python
Date Thu, 11 Oct 2018 18:01:03 GMT
I have parquet files (each self contained) in S3 and I want to read certain
columns into a pandas dataframe without reading the entire object out of
S3.

Is this implemented?  boto3 in python supports reading from offsets in an
S3 object but I wasn't sure anyone has made that work with a parquet file
corresponding to certain columns?

thanks,
Luke

Mime
View raw message