hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4173) If the NameNode has already been formatted, but a QuroumJournal has not, auto-format it on startup
Date Tue, 20 Nov 2012 19:02:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501379#comment-13501379

Colin Patrick McCabe commented on HDFS-4173:

bq. so this would cause the NN to initialize the shared edits dir on standby startup, wouldn't

There's an "if" statement like this:
if (u.getScheme().equals(NNStorage.LOCAL_URI_SCHEME)) {
} else {
   // format

Since the NFS shared edits directory has a LOCAL_URI_SCHEME (it's a path, after all), it won't
be affected by this change.

The question of whether the SNN and NN will both try to auto-format the same (QJM-managed)
{{JournalNode}} quorum when starting up for the first time is more concerning.  We could hook
into {{FSEditLog#initJournalsForWrite}}.  In HA mode, that only gets called later, when a
NN becomes active.
> If the NameNode has already been formatted, but a QuroumJournal has not, auto-format
it on startup
> --------------------------------------------------------------------------------------------------
>                 Key: HDFS-4173
>                 URL: https://issues.apache.org/jira/browse/HDFS-4173
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: journal-node, name-node
>    Affects Versions: 2.0.3-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>         Attachments: HDFS-4173.001.patch
> If we have multiple edit log directories, and some of them are formatted, but others
are not, we format the unformatted ones.  However, when we implemented QuorumJournalManager,
we did not extend this behavior to it.  It makes sense to do this.
> One use case is if you want to add a QuorumJournalManager URI ({{journal://}}) to an
existing {{NameNode}}, without reformatting everything.  There is currently no easy way to
do this, since {{namenode \-format}} will nuke everything, and there's no other way to format
the {{JournalNodes}}.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message