Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
Date: Mon, 4 Mar 2013 23:57:12 +0000 (UTC)
From: "Jay Kreps (JIRA)" <jira@apache.org>
To: dev@kafka.apache.org
Message-ID: <JIRA.12629631.1359429612095.378993.1362441432793@arcas>
In-Reply-To: <JIRA.12629631.1359429612095@arcas>
References: <JIRA.12629631.1359429612095@arcas>
Subject: [jira] [Resolved] (KAFKA-741) Improve log cleaning dedupe buffer
 efficiency
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/KAFKA-741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jay Kreps resolved KAFKA-741.
-----------------------------

    Resolution: Duplicate

This issue is fixed in the patch for KAFKA-739. It removes the duplication using a probing scheme and counts updates correctly.
                
> Improve log cleaning dedupe buffer efficiency
> ---------------------------------------------
>
>                 Key: KAFKA-741
>                 URL: https://issues.apache.org/jira/browse/KAFKA-741
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Jay Kreps
>            Assignee: Jay Kreps
>             Fix For: 0.8.1
>
>
> Two good suggestions:
> 1. Use a probing scheme to increase density without increasing the collision rate
> 2. Only count unique updates to the offset map (i.e. if the key is all zero, don't count it) when computing the load. Dynamically choose the end offset based on when the map is full.
> Would be good to investigate these things.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira