hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reid Chan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18309) Support multi threads in CleanerChore
Date Mon, 20 Nov 2017 13:59:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16259257#comment-16259257

Reid Chan commented on HBASE-18309:

bq. As concerned, it's possible for thread flood happen and drag NN down. And i have an idea
(not quite sure, and need some discussions), but would please allow me to file another jira
for that, it's too squeezed to put into one patch.
HBASE-19306, FYI @Yu Li and anyone has the same concern

> Support multi threads in CleanerChore
> -------------------------------------
>                 Key: HBASE-18309
>                 URL: https://issues.apache.org/jira/browse/HBASE-18309
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>            Reporter: binlijin
>            Assignee: Reid Chan
>         Attachments: HBASE-18309.master.001.patch, HBASE-18309.master.002.patch, HBASE-18309.master.004.patch,
HBASE-18309.master.005.patch, HBASE-18309.master.006.patch, HBASE-18309.master.007.patch,
HBASE-18309.master.008.patch, HBASE-18309.master.009.patch, HBASE-18309.master.010.patch,
HBASE-18309.master.011.patch, space_consumption_in_archive.png
> There is only one thread in LogCleaner to clean oldWALs and in our big cluster we find
this is not enough. The number of files under oldWALs reach the max-directory-items limit
of HDFS and cause region server crash, so we use multi threads for LogCleaner and the crash
not happened any more.

This message was sent by Atlassian JIRA

View raw message