hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "huaxiang sun (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18083) Make large/small file clean thread number configurable in HFileCleaner
Date Sun, 21 May 2017 14:01:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16018838#comment-16018838
] 

huaxiang sun commented on HBASE-18083:
--------------------------------------

Thanks [~carp84]. I believe it is HADOOP-2232. 

{bq}
Pretty fast in our case, and no obvious change in NN metrics after we opened 25 threads for
each (large and small file deleting)
{bq}

With 50 threads, it is fast enough to finish one loop. I think I got the picture together
with HBASE-18084, looking forward to your patch.


> Make large/small file clean thread number configurable in HFileCleaner
> ----------------------------------------------------------------------
>
>                 Key: HBASE-18083
>                 URL: https://issues.apache.org/jira/browse/HBASE-18083
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Yu Li
>            Assignee: Yu Li
>
> Currently we have only one thread for both large and small file cleaning, but when write
pressure is huge we might need more cleaner threads, so we need to make the thread number
configurable.
> We observed more than 1.8PB data in archive directory online due to business access rate
change, and this proposal is one of the necessary changes required.
> Default value of the configurations would still be left to 1 to keep low pressure to
NN for normal case.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message