hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1926) Remove references to StorageDirectory from JournalManager interface
Date Fri, 13 May 2011 03:16:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032809#comment-13032809

Todd Lipcon commented on HDFS-1926:

Hey Ivan. I've been thinking a bit about the NNStorageListener stuff recently.

The original design of this code was to maintain the old behavior which made sure that error
status was linked between storage directories and edits directories -- i.e so that if an image
failed to save in some dir, then we'd make sure to treat those edits as dead too, like you

One of the motivations of HDFS-1073 is that it decouples the edit logs and the images so there's
no longer any coordination. That is to say, even if an image fails to save in a directory
for some reason, it doesn't necessarily mean that we have to immediately stop writing logs.

Can you think of any good reasons why we still need this coupling of error state? I'd be in
favor of getting rid of it altogether since it should simplify a lot of the error-handling
code, and make the edits storage more generic like you said.

> Remove references to StorageDirectory from JournalManager interface
> -------------------------------------------------------------------
>                 Key: HDFS-1926
>                 URL: https://issues.apache.org/jira/browse/HDFS-1926
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-1926.diff
> The JournalManager interface introduced by HDFS-1799 has a getStorageDirectory method
which is out of place in a generic interface. This JIRA removed that call by refactoring the
error handling for FSEditLog. Each EditLogFileOutputStream is now a NNStorageListener and
listens for error on it's containing StorageDirectory. If an error occurs from FSImage, the
stream will be aborted. If the error occurs in FSEditLog, the stream will be aborted and NNStorage
will be notified that the StorageDirectory is no longer valid.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message