hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
Date Wed, 27 Jun 2012 01:09:44 GMT

     [ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Todd Lipcon updated HDFS-3190:

    Attachment: hdfs-3190.txt

Attaching new draft of this patch:

- I agreed with Colin that LongContainingFile was a bad name. I ended up renaming to PersistentLong,
and clarified with JavaDoc what its purpose is. I also made it instantiable as a wrapper which
holds a persisted long value -- this was useful in development of HDFS-3077 in order to hold
the "promised epoch" persistent across restarts.

- Instead of introducing StorageErrorReporter, I just moved the error reporting functionality
up into {{Storage}} instead of {{NNStorage}}. It seems like a generally useful thing -- in
the future we may want to consolidate the error-tracking functionality between the DN and
NN using this mechanism, for example. For now, the {{Storage}} implementation just logs the

- Change TransferFsImage to now take a Storage instead of NNStorage. This is so that in HDFS-3077,
we can download logs into a new {{JNStorage}} class.

- Move {{getFiles()}} from {{NNStorage}} into {{Storage}} since it's also generally useful
and not NN-specific.

- Some minor refactor in {{TransferFsImage]} to make code more re-usable (also used for edits
transfer in HDFS-3077).

While extracting PersistentLong, I also noticed a bug that, if there were an IOE while writing
the file, it would still attempt to close the AtomicFileOutputStream. This could cause the
incompletely written value to get incorrectly "committed". I added a simple "abort" function
for this.
> Simple refactors in existing NN code to assist QuorumJournalManager extension
> -----------------------------------------------------------------------------
>                 Key: HDFS-3190
>                 URL: https://issues.apache.org/jira/browse/HDFS-3190
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Minor
>         Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt
> This JIRA is for some simple refactors in the NN:
> - refactor the code which writes the seen_txid file in NNStorage into a new "LongContainingFile"
utility class. This is useful for the JournalNode to atomically/durably record its last promised
> - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport
interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message