hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18309) Support multi threads in CleanerChore
Date Wed, 08 Nov 2017 04:23:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243351#comment-16243351

Yu Li commented on HBASE-18309:

I think we should focus on {{LogCleaner}} instead of {{CleanerChore}} here (misleading title...),
since it's hard to unify the clean policy for hfile/wal. And in the implementation we're using
in production we made LogCleaner supports online configuration change, which I think is a
must-have (restarting RS is something we better avoid for online/near-line service).

Sorry for the late response since we're busy supporting Singles' Day. Thanks for taking the
task and good on you [~reidchan].

[~aoxiang] please also help review the patch when time allows, thanks.

> Support multi threads in CleanerChore
> -------------------------------------
>                 Key: HBASE-18309
>                 URL: https://issues.apache.org/jira/browse/HBASE-18309
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>            Reporter: binlijin
>            Assignee: Reid Chan
>         Attachments: HBASE-18309.master.001.patch, HBASE-18309.master.002.patch
> There is only one thread in LogCleaner to clean oldWALs and in our big cluster we find
this is not enough. The number of files under oldWALs reach the max-directory-items limit
of HDFS and cause region server crash, so we use multi threads for LogCleaner and the crash
not happened any more.

This message was sent by Atlassian JIRA

View raw message