orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <owen.omal...@gmail.com>
Subject Re: Zstd decoder support
Date Fri, 04 May 2018 20:41:47 GMT
I just upgraded ORC to use aircompressor 0.10. I assume we'll want to move
to 0.11 before we use zstd?

.. Owen

On Fri, May 4, 2018 at 12:49 PM, Dain Sundstrom <dain@iq80.com> wrote:

> The maintained location (and the version we use in prod) is:
> https://github.com/airlift/aircompressor/tree/master/src/
> main/java/io/airlift/compress/zstd
>
> We plan on writing the compressor soon as we need it for our production
> systems.
>
> BTW we are about to do a release that fixes a bug with zstd.
>
> -dain
>
> > On May 4, 2018, at 11:19 AM, Xiening Dai <xndai.git@live.com> wrote:
> >
> > Hi all,
> >
> > I think the major reason that we don’t support zstd compressor today is
> that there’s no native java library currently. But I do see a java
> decompressor in presto code base -
> >
> > https://github.com/prestodb/presto/blob/8f4e5bb9340890f01291ee1b777a1b
> 2b921a90c4/presto-orc/src/main/java/com/facebook/presto/
> orc/zstd/ZstdDecompressor.java
> >
> > I wonder if we can just integrate this so at least we have the
> decompress ability. I believe this is valuable as there are Orc files
> compressed in zstd, including some presto workload, and some of the
> workload in our installation (generated by c++ writer with zstd
> compression).
> >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message