hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1926) Remove references to StorageDirectory from JournalManager interface
Date Fri, 13 May 2011 16:58:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033125#comment-13033125

Todd Lipcon commented on HDFS-1926:

bq. I guess the only problem here would be if write didn't fail immediately, but blocked indefinitely
(if the StorageDirectory pointed to a hard mounted NFS share for example).

Yea, I thought about that as well (it's something we see a lot with customers who forget to
set their NFS mounts to soft with a low timeout).

But, I think, we never know whether we'll see this problem first with an editlog write vs
an image save, so we have to handle this case in both places anyway, perhaps by adding timeouts
to such operations. I think we should deal with that as part of HDFS-1603.

It made a lot of sense before when we needed agreement between images and editlogs about whether
a directory was active, in order to coordinate image/edit rolls. But now that the two are
entirely orthogonal (except for happening to share a directory) I don't see much benefit.

Let me ping Sanjay as well to see if he has an opinion here.

> Remove references to StorageDirectory from JournalManager interface
> -------------------------------------------------------------------
>                 Key: HDFS-1926
>                 URL: https://issues.apache.org/jira/browse/HDFS-1926
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>         Attachments: HDFS-1926-nolistener.diff, HDFS-1926.diff, HDFS-1926.diff
> The JournalManager interface introduced by HDFS-1799 has a getStorageDirectory method
which is out of place in a generic interface. This JIRA removed that call by refactoring the
error handling for FSEditLog. Each EditLogFileOutputStream is now a NNStorageListener and
listens for error on it's containing StorageDirectory. If an error occurs from FSImage, the
stream will be aborted. If the error occurs in FSEditLog, the stream will be aborted and NNStorage
will be notified that the StorageDirectory is no longer valid.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message