hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Toshihiro Suzuki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19478) Utilize multi-get to speed up WAL file checking in BackupLogCleaner
Date Sun, 31 Dec 2017 16:28:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307248#comment-16307248
] 

Toshihiro Suzuki commented on HBASE-19478:
------------------------------------------

I attached a patch that is added new method BackupSystemTable#isWALFilesDeletable() using
multi-get.

> Utilize multi-get to speed up WAL file checking in BackupLogCleaner
> -------------------------------------------------------------------
>
>                 Key: HBASE-19478
>                 URL: https://issues.apache.org/jira/browse/HBASE-19478
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Ted Yu
>            Assignee: Toshihiro Suzuki
>         Attachments: HBASE-19478.patch
>
>
> Currently BackupLogCleaner#getDeletableFiles() issues one Get per WAL file:
> {code}
>       for (FileStatus file : files) {
>         String wal = file.getPath().toString();
>         boolean logInSystemTable = table.isWALFileDeletable(wal);
> {code}
> This is rather inefficient considering the number of WAL files in production can get
quite large.
> We should use multi-get to reduce the number of calls to backup table (which normally
resides on another server).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message