hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4173) If the NameNode has already been formatted, but a QuroumJournal has not, auto-format it on startup
Date Tue, 20 Nov 2012 18:16:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501346#comment-13501346
] 

Todd Lipcon commented on HDFS-4173:
-----------------------------------

Hey Colin. I'm not sure this is the right place for this new code -- the {{initJournals}}
function gets called even when the journals are being initialized for _read_ -- so this would
cause the NN to initialize the shared edits dir on standby startup, wouldn't it? ie if we
configure for HA and don't run the explicit -initializeSharedEdits, I don't think it's appropriate
to auto-format, since it could end up with a race where both nodes try to format and end up
formatting each other's dirs. Am I missing something?

At the least we should probably add a comment in {{FSImage.recoverTransitionRead}} which points
out that non-file storage directories will get formatted when the edit log is initialized.
                
> If the NameNode has already been formatted, but a QuroumJournal has not, auto-format
it on startup
> --------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-4173
>                 URL: https://issues.apache.org/jira/browse/HDFS-4173
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: journal-node, name-node
>    Affects Versions: 2.0.3-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>         Attachments: HDFS-4173.001.patch
>
>
> If we have multiple edit log directories, and some of them are formatted, but others
are not, we format the unformatted ones.  However, when we implemented QuorumJournalManager,
we did not extend this behavior to it.  It makes sense to do this.
> One use case is if you want to add a QuorumJournalManager URI ({{journal://}}) to an
existing {{NameNode}}, without reformatting everything.  There is currently no easy way to
do this, since {{namenode \-format}} will nuke everything, and there's no other way to format
the {{JournalNodes}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message