hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2866) Standby does not start up due to a gap in transaction id
Date Wed, 01 Feb 2012 02:18:56 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197515#comment-13197515

Aaron T. Myers commented on HDFS-2866:

bq. don't we need a shared copy of fsimage?

Nope, and configuring it as such would be an error.

bq. If it is kept in local dirs, there would be a problem with two NNs formatting unless something
like metadata copy is done from local dirs of NN1 to NN2.

The intention is that when bootstrapping an HA cluster, one must format one NN and then copy
the initially-empty metadata to the other NN. This is similar to bootstrapping other HA systems
with persistent data, e.g. MySQL. See this JIRA which aims to improve this bootstrapping process:

bq. Also, working with local image of fsimage might make it difficult to catch issues if there
is divergence.

How so? The standby NN performs checkpoints and thus will periodically download fsimage files
from the active NN. Also note that the edits and image files include the transaction IDs they
> Standby does not start up due to a gap in transaction id
> --------------------------------------------------------
>                 Key: HDFS-2866
>                 URL: https://issues.apache.org/jira/browse/HDFS-2866
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha
>    Affects Versions: HA branch (HDFS-1623)
>            Reporter: Hari Mankude
>            Priority: Critical
> Standby notices a gap in the transaction id in the shared.edits directory. The transactions
in dfs.edits.dir does not seem to have the gap. The gap happens during a failover.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message