kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Cheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6175) AbstractIndex should cache index file to avoid unnecessary disk access during resize()
Date Mon, 06 Nov 2017 04:38:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16239899#comment-16239899
] 

James Cheng commented on KAFKA-6175:
------------------------------------

Do you have any estimates of how much time is saved, similar to your benchmarks in https://issues.apache.org/jira/browse/KAFKA-6172?

> AbstractIndex should cache index file to avoid unnecessary disk access during resize()
> --------------------------------------------------------------------------------------
>
>                 Key: KAFKA-6175
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6175
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Dong Lin
>            Assignee: Dong Lin
>             Fix For: 1.0.1
>
>
> Currently when we shutdown a broker, we will call AbstractIndex.resize() for all segments
on the broker, regardless of whether the log segment is active or not. AbstractIndex.resize()
incurs raf.setLength(), which is expensive because it accesses disks. If we do a threaddump
during either LogManger.shutdown() or LogManager.loadLogs(), most threads are in RUNNABLE
state at java.io.RandomAccessFile.setLength().
> This patch intends to speed up broker startup and shutdown time by skipping AbstractIndex.resize()
for inactive log segments.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message