kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dongjinleekr <...@git.apache.org>
Subject [GitHub] kafka pull request #2267: KAFKA-4514: Add Codec for ZStandard Compression
Date Fri, 16 Dec 2016 09:52:18 GMT
GitHub user dongjinleekr opened a pull request:


    KAFKA-4514: Add Codec for ZStandard Compression

    Hello. This PR resolves [KAFKA-4514: Add Codec for ZStandard Compression](https://issues.apache.org/jira/browse/KAFKA-4514).
Please have a look when you are free. Since I am a total newbie of Apache Kafka, feel free
to point out the deficiencies.
    Add to the feature itself, I have a question: **Should we support an option for ZStandard
compression level?**
    According to [ZStandard official documentation](https://github.com/facebook/zstd/blob/dev/lib/zstd.h),
it supports compression level of 1 ~ 22. Because of that, [Hadoop added a new configuration
option named "io.compression.codec.zstd.level", whose default value is 3](https://issues.apache.org/jira/browse/HADOOP-13578).
In this PR, I configured the compression level to 1 as a temporary one but wondering following
    - Should we provide a configurable option?
    - Would it better to change the default value, from 1 to another one?
    I am looking forward to your advice. Thanks.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dongjinleekr/kafka feature/KAFKA-4514

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2267
commit aa751148b75cac9062b13b1115738a595556dd39
Author: Lee Dongjin <dongjin@apache.org>
Date:   2016-12-16T08:50:20Z

    Add zstandard dependency

commit 22fe8d64ffbc2d2725bdfb2166f1ecdc75a96c8d
Author: Lee Dongjin <dongjin@apache.org>
Date:   2016-12-16T08:51:20Z

    Add ZStdCompressionCodec

commit ab535e9f0a18132e35dcbef032dcde320204e12b
Author: Lee Dongjin <dongjin@apache.org>
Date:   2016-12-16T08:59:31Z

    Add CompressionType.ZSTD and update ProducerConfig, MemoryRecordsBuilder

commit 87c3e803ab30344a9834c19075a3ebe617cf5787
Author: Lee Dongjin <dongjin@apache.org>
Date:   2016-12-16T09:00:55Z

    Update CompressionCodec & CompressionFactory

commit 18af6439f1fad6811783da43d0a0f475164853c8
Author: Lee Dongjin <dongjin@apache.org>
Date:   2016-12-16T09:03:58Z

    Add tests for ZStandard

commit 41ae0ac1ac809822712eb609726db00d00e0f36d
Author: Lee Dongjin <dongjin@apache.org>
Date:   2016-12-16T09:14:28Z

    Add documentation comments


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message