hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-6545) Finalizing rolling upgrade can make NN unavailable for a long duration
Date Mon, 16 Jun 2014 21:26:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032962#comment-14032962
] 

Kihwal Lee edited comment on HDFS-6545 at 6/16/14 9:25 PM:
-----------------------------------------------------------

I think we can skip {{saveNamespace()}}. Presence of "finalize" op in a consumable edit segment
should not cause any trouble, since any subsequent rolling upgrade will create a rollback
image with txid past the op.  The only potential inconvenience is that if ANN or SBN needs
to be restarted before the new version saving a checkpoint for some reason, it may need "-rollingUpgrade
started" start-up option if there was a layout version change.  But this can also happen on
SBN with the current code.    

In summary, removing {{saveNamespace()}} from {{finalizeRollingUpgrade()}} seems reasonable.



was (Author: kihwal):
I think we can skip {{saveNamespace()}}. Presence of "finalize" op in a consumable edit segment
should not cause any trouble, since any subsequent rolling upgrade will create a rollback
image with txid past the op.  The only potential inconvenience is that if ANN or SBN needs
to be restarted before the new version saving a checkpoint for some reason, it may need "-rollingUpgrade
started" start-up option if there was a layout version change.  But this can also happen on
SBN with the current code.  If SBN was not up when ANN is finalizing an upgrade, SBN may need
to be started with "-rollingUpgrade started" in order for it to read the rollback image saved
by the old version. Or it will need to be bootstrapped fresh in order to download the new
fsimage created only on ANN through {{saveNamespace()}}.  

In summary, removing {{saveNamespace()}} from {{finalizeRollingUpgrade()}} seems reasonable.


> Finalizing rolling upgrade can make NN unavailable for a long duration
> ----------------------------------------------------------------------
>
>                 Key: HDFS-6545
>                 URL: https://issues.apache.org/jira/browse/HDFS-6545
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.4.0
>            Reporter: Kihwal Lee
>            Priority: Critical
>
> In {{FSNamesystem#finalizeRollingUpgrade()}}, {{saveNamespace()}} is directly called.
 For name nodes with a big name space, it can take minutes to save a new fsimage.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message