orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dain Sundstrom <d...@iq80.com>
Subject Re: Zstd decoder support
Date Sat, 05 May 2018 02:56:36 GMT
0.11 is released.

-dain

Sent from my iPhone

> On May 4, 2018, at 1:41 PM, Owen O'Malley <owen.omalley@gmail.com> wrote:
>
> I just upgraded ORC to use aircompressor 0.10. I assume we'll want to move
> to 0.11 before we use zstd?
>
> .. Owen
>
>> On Fri, May 4, 2018 at 12:49 PM, Dain Sundstrom <dain@iq80.com> wrote:
>>
>> The maintained location (and the version we use in prod) is:
>> https://github.com/airlift/aircompressor/tree/master/src/
>> main/java/io/airlift/compress/zstd
>>
>> We plan on writing the compressor soon as we need it for our production
>> systems.
>>
>> BTW we are about to do a release that fixes a bug with zstd.
>>
>> -dain
>>
>>> On May 4, 2018, at 11:19 AM, Xiening Dai <xndai.git@live.com> wrote:
>>>
>>> Hi all,
>>>
>>> I think the major reason that we don’t support zstd compressor today is
>> that there’s no native java library currently. But I do see a java
>> decompressor in presto code base -
>>>
>>> https://github.com/prestodb/presto/blob/8f4e5bb9340890f01291ee1b777a1b
>> 2b921a90c4/presto-orc/src/main/java/com/facebook/presto/
>> orc/zstd/ZstdDecompressor.java
>>>
>>> I wonder if we can just integrate this so at least we have the
>> decompress ability. I believe this is valuable as there are Orc files
>> compressed in zstd, including some presto workload, and some of the
>> workload in our installation (generated by c++ writer with zstd
>> compression).
>>>
>>>
>>
>>

Mime
View raw message