hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-7973) DistributedFileSystem close has severe consequences
Date Tue, 17 Jan 2012 23:45:39 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daryn Sharp updated HADOOP-7973:
--------------------------------

    Attachment: HADOOP-7973-2.patch

There's opposition to disabling the cache, so here's an alternate approach to the problem.
 I'm again only posting untested code for community comment.

Basically if the FS is cached, then it isn't closed until removed from the cache.  This allows
the cache to remain enabled so {{FileSystem.closeAll}}, {{FileSystem.closeAllForUGI}}, etc
continue to work as intended: boost performance and not let objects leak.  If caching is disabled,
then the object is immediately closed.

Please critique.
                
> DistributedFileSystem close has severe consequences
> ---------------------------------------------------
>
>                 Key: HADOOP-7973
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7973
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 1.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HADOOP-7973-2.patch, HADOOP-7973.patch
>
>
> The way {{FileSystem#close}} works is very problematic.  Since the {{FileSystems}} are
cached, any {{close}} by any caller will cause problems for every other reference to it. 
Will add more detail in the comments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message