Mailing-List: contact jira-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@kafka.apache.org
Date: Tue, 11 Jul 2017 19:38:00 +0000 (UTC)
From: "Jason Aliyetti (JIRA)" <jira@apache.org>
To: jira@kafka.apache.org
Message-ID: <JIRA.13086363.1499801864000.220961.1499801880221@Atlassian.JIRA>
In-Reply-To: <JIRA.13086363.1499801864000@Atlassian.JIRA>
References: <JIRA.13086363.1499801864000@Atlassian.JIRA> <JIRA.13086363.1499801864027@jira-lw-us.apache.org>
Subject: [jira] [Created] (KAFKA-5582) Log compaction with preallocation
 enabled does not trim segments
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 11 Jul 2017 19:53:40 -0000

Jason Aliyetti created KAFKA-5582:
-------------------------------------

             Summary: Log compaction with preallocation enabled does not trim segments
                 Key: KAFKA-5582
                 URL: https://issues.apache.org/jira/browse/KAFKA-5582
             Project: Kafka
          Issue Type: Bug
    Affects Versions: 0.10.1.1
         Environment: Linux, Windows
            Reporter: Jason Aliyetti


Unexpected behavior occurs when a topic is configured to preallocate files and has a retention policy of compact.

When log compaction runs, the cleaner attempts to gather groups of segments to consolidate based on the max segment size.  
When preallocation is enabled all segments are that size and thus each individual segment is considered for compaction.

When compaction does occur, the resulting cleaned file is sized based on that same configuration.  This means that you can have very large files on disk that contain little or no data which partly defeats the point of compacting. 

The log cleaner should trim these segments to free up disk space.  That way they would free up disk space and be able to be further compacted on subsequent runs.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)