hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14621) ReplicationLogCleaner gets stuck when a regionserver crashes
Date Sat, 17 Oct 2015 03:25:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14961708#comment-14961708
] 

Hudson commented on HBASE-14621:
--------------------------------

SUCCESS: Integrated in HBase-0.98 #1160 (See [https://builds.apache.org/job/HBase-0.98/1160/])
HBASE-14621 ReplicationLogCleaner gets stuck when a regionserver crashes (tedyu: rev 50968d70af5c427a61facd4da9744f565d2b56cd)
* hbase-server/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java


> ReplicationLogCleaner gets stuck when a regionserver crashes
> ------------------------------------------------------------
>
>                 Key: HBASE-14621
>                 URL: https://issues.apache.org/jira/browse/HBASE-14621
>             Project: HBase
>          Issue Type: Bug
>          Components: Replication
>            Reporter: Ashu Pachauri
>            Assignee: Ashu Pachauri
>            Priority: Critical
>             Fix For: 2.0.0, 1.2.0, 1.0.3, 1.1.3, 0.98.16
>
>         Attachments: HBASE-14621-1.patch, HBASE-14621.patch
>
>
> The ReplicationLogCleaner has a bug that makes it get stuck in an infinite loop when
a regionserver crashes. This bug was introduced in the fix for HBASE-12865; which makes sure
that the loadWALsFromQueues method attempts a retry whenever the replication node's cversion
is changed in the middle of loading the replication queue for the regionservers. However,
if this scenario actually happens (a regionserver crash in the middle of the operation), it
will get stuck in an infinite loop.
> It has very serious ramifications because the old WALs are not cleaned up because of
this and in a high load environment, the file count in the oldWALs directory soon exceeds
the inode limit and the cluster goes down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message