hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7973) DistributedFileSystem close has severe consequences
Date Sun, 15 Jan 2012 06:02:39 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186424#comment-13186424

Harsh J commented on HADOOP-7973:

(Coming in late, but just to add my thoughts…)

I've seen this bite users as well but its more so cause they do not understand how to use
the FS objects than anything else:

bq. A MR task that uses FsShell. The shell opens a DFS, performs it's action, and the shell
will close the DFS. Now the MR input stream close to that same fileystem will fail.

Is FsShell a publicly supported API now?

bq. User map task code that opens the default filesystem and subsequently closes it. MR input
stream close will fail.

Users should not be closing the FS handle. Users shall open/close streams they use. Is that
not the right thing to do?
> DistributedFileSystem close has severe consequences
> ---------------------------------------------------
>                 Key: HADOOP-7973
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7973
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 1.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HADOOP-7973.patch
> The way {{FileSystem#close}} works is very problematic.  Since the {{FileSystems}} are
cached, any {{close}} by any caller will cause problems for every other reference to it. 
Will add more detail in the comments.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message