hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18084) Improve CleanerChore to clean from directory which consumes more disk space
Date Sun, 21 May 2017 07:52:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16018729#comment-16018729

Yu Li commented on HBASE-18084:

bq. 'else' can be omitted since return is called in the previous if block.
Yep, maybe simply
return (f1ConsumedSpace > f2ConsumedSpace) ? -1
            : (f1ConsumedSpace < f2ConsumedSpace ? 1 : 0);
Let me make the change.

bq. The map is declared in the comparator which is passed dirs List. How many directories
would find their cached lengths ?
The directories might be few, but the {{fs.getContentSummary}} call is time consuming if there're
many files in the directory like in our case, so I added this map as a cache.

> Improve CleanerChore to clean from directory which consumes more disk space
> ---------------------------------------------------------------------------
>                 Key: HBASE-18084
>                 URL: https://issues.apache.org/jira/browse/HBASE-18084
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Yu Li
>            Assignee: Yu Li
>         Attachments: HBASE-18084.patch
> Currently CleanerChore cleans the directory in dictionary order, rather than from the
directory with largest space usage. And when data abnormally accumulated to some huge volume
in archive directory, the cleaning speed might not be enough.
> This proposal is another improvement working together with HBASE-18083 to resolve our
online issue (archive dir consumed more than 1.8PB SSD space)

This message was sent by Atlassian JIRA

View raw message