kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edoardo Comar (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-7761) CLONE - Add broker configuration to set minimum value for segment.bytes and segment.ms
Date Mon, 24 Dec 2018 10:40:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Edoardo Comar resolved KAFKA-7761.
----------------------------------
    Resolution: Duplicate

if there is a valid reason for a duplicate, feel free to repoen

> CLONE - Add broker configuration to set minimum value for segment.bytes and segment.ms
> --------------------------------------------------------------------------------------
>
>                 Key: KAFKA-7761
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7761
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Chinmay Patil
>            Priority: Major
>              Labels: kip, newbie
>
> If someone set segment.bytes or segment.ms at topic level to a very small value (e.g.
segment.bytes=1000 or segment.ms=1000), Kafka will generate a very high number of segment
files. This can bring down the whole broker due to hitting the maximum open file (for log)
or maximum number of mmap-ed file (for index).
> To prevent that from happening, I would like to suggest adding two new items to the
broker configuration:
>  * min.topic.segment.bytes, defaults to 1048576: The minimum value for segment.bytes. When
someone sets topic configuration segment.bytes to a value lower than this, Kafka throws an
error INVALID VALUE.
>  * min.topic.segment.ms, defaults to 3600000: The minimum value for segment.ms. When
someone sets topic configuration segment.ms to a value lower than this, Kafka throws an
error INVALID VALUE.
> Thanks
> Badai



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message