kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mathias Herberts (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-235) Add a 'log.file.age' configuration parameter to force rotation of log files after they've reached a certain age
Date Thu, 05 Jan 2012 14:47:39 GMT
Add a 'log.file.age' configuration parameter to force rotation of log files after they've reached
a certain age
---------------------------------------------------------------------------------------------------------------

                 Key: KAFKA-235
                 URL: https://issues.apache.org/jira/browse/KAFKA-235
             Project: Kafka
          Issue Type: New Feature
          Components: core
    Affects Versions: 0.7
            Reporter: Mathias Herberts


The Kafka client has the ability to start consuming at an offset before or after a given point
in time. The granularity of this offset is the log file as the Kafka servers do not keep track
of arrival time of various messages.

This means that the granularity of offsets relative to time depends on arrival rate of messages
and thus of log file rotation. A topic with lots of messages will have its log files rotated
very often (thus each spans a short time interval) whereas a topic with very few messages
might not see its log files rotated for hours.

In order to circumvent this granularity disparity, having a parameter that would force log
file rotation after a certain delay (xxx ms) would allow for pretty much constant time granularity
to be available at the cost of more file descriptor being used.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message