orc-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhiyuan Dong <zhiyuan.d...@gmail.com>
Subject Re: access entire column in ORC files
Date Sat, 26 Jan 2019 01:35:59 GMT
Thanks Xiening!!

A follow-up  question :

suppose I have an orc files having many columns,

in the first pass, I read the first column from start to end to find out
which are the subset of the rows that I need to extract.

now, when I do a 2nd pass, for the rest of columns, is there any efficient
way that I can only extract the row positions that I identified in the
first pass ?

what I am doing now is to extract the rest of columns, batch by batch, and
only extract those rows identified by the first pass, but not sure if this
is an efficient way.

Many thanks!!

Best,

Zhiyuan

Mime
View raw message