kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5236) Regression in on-disk log size when using Snappy compression with 0.8.2 log message format
Date Fri, 02 Jun 2017 20:22:04 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035353#comment-16035353

ASF GitHub Bot commented on KAFKA-5236:

Github user asfgit closed the pull request at:


> Regression in on-disk log size when using Snappy compression with 0.8.2 log message format
> ------------------------------------------------------------------------------------------
>                 Key: KAFKA-5236
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5236
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions:
>            Reporter: Nick Travers
>            Assignee: Ismael Juma
>              Labels: regression
>             Fix For:
> We recently upgraded our brokers in our production environments from to
and we've noticed a sizable regression in the on-disk .log file size. For some deployments
the increase was as much as 50%.
> We run our brokers with the 0.8.2 log message format version. The majority of our message
volume comes from 0.10.x Java clients sending messages encoded with the Snappy codec.
> Some initial testing only shows a regression between the two versions when using Snappy
compression with a log message format of 0.8.2.
> I also tested 0.10.x log message formats as well as Gzip compression. The log sizes do
not differ in this case, so the issue seems confined to 0.8.2 message format and Snappy compression.
> A git-bisect lead me to this commit, which modified the server-side implementation of
> https://github.com/apache/kafka/commit/67f1e5b91bf073151ff57d5d656693e385726697
> Here's the PR, which has more context:
> https://github.com/apache/kafka/pull/2140
> Here is a link to the test I used to re-producer this issue:
> https://github.com/nicktrav/kafka/commit/68e8db4fa525e173651ac740edb270b0d90b8818
> cc: [~hachikuji] [~junrao] [~ijuma] [~guozhang] (tagged on original PR)

This message was sent by Atlassian JIRA

View raw message