hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-7973) DistributedFileSystem close has severe consequences
Date Wed, 18 Jan 2012 23:58:39 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Daryn Sharp updated HADOOP-7973:

    Attachment: HADOOP-7973-3.patch

Idea is basically to allow MR to be backward-compatible by maintaining cached FS objects that
are distinct from the FS objects used by user code.  That's exactly how it used to work due
to the DFS port-stripping bug.

I pulled in minimal changes from trunk to allow a unique id in the FS cache key.  I didn't
pull in the new APIs, but rather smuggled a unique id in via a config setting. It's not the
cleanest, but I'm open to alternatives.

I've probably missed a few spots in the MR framework, but I'd like comments on the approach
before I go further.
> DistributedFileSystem close has severe consequences
> ---------------------------------------------------
>                 Key: HADOOP-7973
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7973
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 1.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HADOOP-7973-2.patch, HADOOP-7973-3.patch, HADOOP-7973.patch
> The way {{FileSystem#close}} works is very problematic.  Since the {{FileSystems}} are
cached, any {{close}} by any caller will cause problems for every other reference to it. 
Will add more detail in the comments.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message