hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7973) DistributedFileSystem close has severe consequences
Date Fri, 13 Jan 2012 20:44:39 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185837#comment-13185837

Suresh Srinivas commented on HADOOP-7973:

bq. Unlike other filesystems, DFS used to strip the default port from its uris. Ie.FileSystem.get("hdfs://host:port").getUri()
did not return "hdfs://host:port". It returned "hdfs://host".
That still should have resulted in this issue right? May be I do not understand this correctly.

bq. two wrongs make a right
Either by fluke or not, the system worked. This is the reason why I am very cautious about
changes that could have unforeseen outcomes. Because all the interactions and corner cases
are not understood.

bq. so is it better to "fix" the public api, or tell users don't use the public api?
Users are already using those APIs and the API has certain behavior. Turning off that functionality
is not backward compatible.

I think of only one solution. See if long running clients are creating a lot of file systems.
If not it should be safe to turn off cache. BTW I remember conversations with Dhruba where
he had indicated they do not use file system cache.

> DistributedFileSystem close has severe consequences
> ---------------------------------------------------
>                 Key: HADOOP-7973
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7973
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 1.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HADOOP-7973.patch
> The way {{FileSystem#close}} works is very problematic.  Since the {{FileSystems}} are
cached, any {{close}} by any caller will cause problems for every other reference to it. 
Will add more detail in the comments.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message