hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10244) Avoid accumulation of IBRs for SNN when the standby is down for more than expected time
Date Wed, 02 Aug 2017 16:51:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111272#comment-16111272

Daryn Sharp commented on HDFS-10244:

#2 is not necessary.  The are no "important IBRs" that warrant a DN excessively buffering
IBRs, let alone assuming it knows the NN's heartbeat expiry.  The DN can choose to discard
IBRs at any time and simply send a FBR either directly, or via willful re-registration.  The
latter may be easier to accomplish, ex. trick the DN into thinking it hasn't registered.

> Avoid accumulation of IBRs for SNN when the standby is down for more than expected time
> ---------------------------------------------------------------------------------------
>                 Key: HDFS-10244
>                 URL: https://issues.apache.org/jira/browse/HDFS-10244
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Brahma Reddy Battula
>            Assignee: Brahma Reddy Battula
> As per discussion in HDFS-9917 and [comment|https://issues.apache.org/jira/browse/HDFS-9917?focusedCommentId=15213947&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15213947]
from [~vinayrpet] 
>  Avoid accumulation of IBRs when the standby is down for long time, can we consider as
> 1. IBRs for StandbyNN can have a threshold ( say 100K or 1Million IBRs ).
> 2. Also not to loose any important IBRs, IBRs can be cleared when "the threshold is reached
AND 'lastIBR' is more than 'heartbeatExpiryInterval'. i.e. DataNode is considered dead in
Namenode side". In that case, for sure re-Register() will be called on reconnection to running
NameNode (if any).

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message