Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Fri, 13 Jan 2012 20:44:39 +0000 (UTC)
From: "Suresh Srinivas (Commented) (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: 
 <1773439277.39074.1326487479620.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1401268979.38212.1326472840490.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HADOOP-7973) DistributedFileSystem close has
 severe consequences
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185837#comment-13185837 ] 

Suresh Srinivas commented on HADOOP-7973:
-----------------------------------------

bq. Unlike other filesystems, DFS used to strip the default port from its uris. Ie.FileSystem.get("hdfs://host:port").getUri() did not return "hdfs://host:port". It returned "hdfs://host".
That still should have resulted in this issue right? May be I do not understand this correctly.

bq. two wrongs make a right
Either by fluke or not, the system worked. This is the reason why I am very cautious about changes that could have unforeseen outcomes. Because all the interactions and corner cases are not understood.

bq. so is it better to "fix" the public api, or tell users don't use the public api?
Users are already using those APIs and the API has certain behavior. Turning off that functionality is not backward compatible.

I think of only one solution. See if long running clients are creating a lot of file systems. If not it should be safe to turn off cache. BTW I remember conversations with Dhruba where he had indicated they do not use file system cache.

                
> DistributedFileSystem close has severe consequences
> ---------------------------------------------------
>
>                 Key: HADOOP-7973
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7973
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 1.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HADOOP-7973.patch
>
>
> The way {{FileSystem#close}} works is very problematic.  Since the {{FileSystems}} are cached, any {{close}} by any caller will cause problems for every other reference to it.  Will add more detail in the comments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira