orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <omal...@apache.org>
Subject Re: ORC Stripe Skip Using Stripe Level Index
Date Tue, 24 Jan 2017 22:50:09 GMT
That is my fault. We just haven't ported that part of the functionality
over yet. Hive's OrcInputFormat has a lot of complexity that most users
don't need or want. (It's types such as OrcStruct also don't actually work
as Writables, which causes users outside of Hive problems.) The
orc-mapreduce's types do work as Writables and thus work better outside of
Hive. That said, no one has ported the split elimination yet.

.. Owen

On Tue, Jan 24, 2017 at 2:45 PM, Lawan Subba <
lawansubba.mailinglist@gmail.com> wrote:

> Hi Gopal,
>
> Thank you for the quick reply.
>
> I am new to open source projects,  can you also tell me why this
> functionality is missing from the github repository for Apache ORC.
>
> Regards,
> Lawan Subba
>
> On Tue, Jan 24, 2017 at 8:26 PM, Gopal Vijayaraghavan <gopalv@apache.org>
> wrote:
>
> >
> > >    I can see that row indices are being used to select only rowgroups
> > that
> > >    satisfy a search predicate in
> > …
> > >   But, I cannot find where and if the stripe level indices are being
> > used?
> >
> > https://github.com/apache/hive/blob/master/ql/src/java/
> > org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L2031
> >
> > Read through isStripeSatisfyPredicate and pickStripesInternal.
> >
> > Cheers,
> > Gopal
> >
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message