hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hubert Zhang <hzh...@pivotal.io>
Subject Re: [Propose] New PXF profile optimized for ORC (predicate pushdown)
Date Thu, 07 Jul 2016 03:07:54 GMT
+1 for lazy reader, It can save a lot of decompression and
deserialization(CPU bound) time.

On Wed, Jul 6, 2016 at 7:00 AM, Roman Shaposhnik <roman@shaposhnik.org>
wrote:

> On Tue, Jul 5, 2016 at 12:01 PM, Shivram Mani <shivram.mani@gmail.com>
> wrote:
> > I've created the following jira HAWQ-866
> > <https://issues.apache.org/jira/browse/HAWQ-886> which is focussed on
> > improving/enhancing the existing PXF profile to read ORC files. The goal
> is
> > to make use of the underlying ORC reader's capability of supporting
> > predicate push-down among others.
> >
> > Presto has also contributed an alternative ORC reader which provides both
> > predicate push down and Lazy reads
> >
> https://code.facebook.com/posts/370832626374903/even-faster-data-at-the-speed-of-presto-orc/
> > .
> >
> > Will be evaluating both the options as part of this effort.
>
> Great to see this effort! Do you plan to come up with any kind of
> benchmark to
> be able to compare the native ORC reader vs. PXF ORC reader performance
> and capabilities?
>
> Or does it really all just boil down to TPC?
>
> Thanks,
> Roman.
>



-- 
Thanks

Hubert Zhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message