kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sriram Subramanian (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-671) DelayedProduce requests should not hold full producer request data
Date Tue, 15 Jan 2013 02:10:12 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sriram Subramanian updated KAFKA-671:

    Attachment: outOfMemFix-v1.patch

This is one possible fix. The fix would largely depend on the decision for Kafka 703 but I
just wanted one possible fix to be available to make better decision on the other bug.

1. We now create a map of topic partion -> messagesizes and DelayedProduce uses this instead
of the entire data payload. 
2. We reset the data payload at the end of HandleProduceRequest. 

The only way I have found to reset the immutable maps is to make them vars (Other option is
to convert them to mutable maps but that involves a ton of code changes (all the place where
the data objects are passed around need to be changed to mutable maps). Let me know of a better
option in Scala.

The expectation is that the data object will not be used after the reset method is called.
> DelayedProduce requests should not hold full producer request data
> ------------------------------------------------------------------
>                 Key: KAFKA-671
>                 URL: https://issues.apache.org/jira/browse/KAFKA-671
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8
>            Reporter: Joel Koshy
>            Assignee: Sriram Subramanian
>            Priority: Blocker
>             Fix For: 0.8.1
>         Attachments: outOfMemFix-v1.patch
> Per summary, this leads to unnecessary memory usage.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message